lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-ID: <1423986414.5538.8.camel@intel.com>
Date:	Sun, 15 Feb 2015 15:46:54 +0800
From:	Huang Ying <ying.huang@...el.com>
To:	Jason Low <jason.low2@...com>
Cc:	Peter Zijlstra <peterz@...radead.org>,
	LKML <linux-kernel@...r.kernel.org>, LKP ML <lkp@...org>
Subject: [LKP] [mutex] 871a6bb4916: -1.8% will-it-scale.per_process_ops,
 -98.3% will-it-scale.time.voluntary_context_switches, +209.6%
 will-it-scale.time.involuntary_context_switches

FYI, we noticed the below changes on

git://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git locking/core
commit 871a6bb4916fef3123b6ff749b0dc82680fb0d2a ("mutex: In mutex_spin_on_owner(), return true when owner changes")


testbox/testcase/testparams: wsm/will-it-scale/performance-writeseek3

e07e0d4cb0c4bfe8  871a6bb4916fef3123b6ff749b  
----------------  --------------------------  
         %stddev     %change         %stddev
             \          |                \  
  24972759 ±  2%     -98.3%     417134 ±  9%  will-it-scale.time.voluntary_context_switches
      2223 ± 49%    +209.6%       6884 ± 10%  will-it-scale.time.involuntary_context_switches
       542 ± 32%     +91.3%       1037 ±  0%  will-it-scale.time.system_time
       186 ± 30%     +86.3%        347 ±  0%  will-it-scale.time.percent_of_cpu_this_job_got
     26.11 ±  5%     -22.7%      20.18 ±  2%  will-it-scale.time.user_time
      0.09 ±  1%     -18.2%       0.07 ±  1%  will-it-scale.scalability
    783528 ±  0%      -1.8%     769550 ±  0%  will-it-scale.per_process_ops
   6038710 ±  9%     -99.4%      34622 ± 29%  sched_debug.cpu#8.nr_switches
    354318 ± 12%     -88.4%      41041 ±  3%  softirqs.SCHED
      1.67 ± 14%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.pick_next_task_fair.__sched_text_start.schedule.schedule_preempt_disabled.cpu_startup_entry
      1.82 ±  9%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.dequeue_task_fair.dequeue_task.deactivate_task.__sched_text_start.schedule
      2.09 ±  6%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles._raw_spin_lock.try_to_wake_up.wake_up_process.__mutex_unlock_slowpath.mutex_unlock
      2.05 ±  9%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.dequeue_task.deactivate_task.__sched_text_start.schedule.schedule_preempt_disabled
      2.07 ±  9%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.deactivate_task.__sched_text_start.schedule.schedule_preempt_disabled.__mutex_lock_slowpath
      2.45 ± 11%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.__sched_text_start.schedule.schedule_preempt_disabled.cpu_startup_entry.start_secondary
      3.72 ±  3%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.activate_task.ttwu_do_activate.try_to_wake_up.wake_up_process.__mutex_unlock_slowpath
   1033400 ±  5%     -89.4%     109500 ± 30%  sched_debug.cpu#3.ttwu_count
    975947 ±  2%     -88.8%     109094 ± 30%  sched_debug.cpu#3.sched_goidle
     12.27 ± 10%    +492.7%      72.73 ±  1%  perf-profile.cpu-cycles.mutex_optimistic_spin.__mutex_lock_slowpath.mutex_lock.generic_file_write_iter.new_sync_write
      3.22 ± 26%   +1718.0%      58.50 ±  1%  perf-profile.cpu-cycles.osq_lock.mutex_optimistic_spin.__mutex_lock_slowpath.mutex_lock.generic_file_write_iter
      3.22 ± 10%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles._raw_spin_unlock_irqrestore.__hrtimer_start_range_ns.hrtimer_start_range_ns.tick_nohz_restart.tick_nohz_idle_exit
      4.29 ±  9%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.__sched_text_start.schedule.schedule_preempt_disabled.__mutex_lock_slowpath.mutex_lock
      4.54 ±  2%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.ttwu_do_activate.constprop.85.try_to_wake_up.wake_up_process.__mutex_unlock_slowpath.mutex_unlock
      4.02 ±  9%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.__hrtimer_start_range_ns.hrtimer_start_range_ns.tick_nohz_restart.tick_nohz_idle_exit.cpu_startup_entry
   1953046 ±  2%     -88.7%     219857 ± 30%  sched_debug.cpu#3.sched_count
     15.15 ±  2%     -84.0%       2.42 ± 36%  perf-profile.cpu-cycles.__mutex_unlock_slowpath.mutex_unlock.generic_file_write_iter.new_sync_write.vfs_write
      4.39 ± 10%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.schedule.schedule_preempt_disabled.__mutex_lock_slowpath.mutex_lock.generic_file_write_iter
      4.41 ±  9%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.schedule_preempt_disabled.__mutex_lock_slowpath.mutex_lock.generic_file_write_iter.new_sync_write
      6.24 ± 15%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.tick_nohz_stop_sched_tick.__tick_nohz_idle_enter.tick_nohz_idle_enter.cpu_startup_entry.start_secondary
      6.69 ± 15%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.__tick_nohz_idle_enter.tick_nohz_idle_enter.cpu_startup_entry.start_secondary
      6.85 ± 15%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.tick_nohz_idle_enter.cpu_startup_entry.start_secondary
 1.372e+08 ± 14%     -95.4%    6291721 ± 10%  cpuidle.C1-NHM.time
   1005704 ± 29%     -97.1%      28896 ±  3%  cpuidle.C1-NHM.usage
  9.11e+08 ± 13%     -98.8%   10867422 ± 11%  cpuidle.C3-NHM.time
   3887769 ± 14%     -99.3%      28556 ±  8%  cpuidle.C3-NHM.usage
   1510725 ± 22%     -95.9%      62458 ±  1%  cpuidle.C6-NHM.usage
      0.78 ± 32%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles._raw_spin_unlock_irqrestore.__hrtimer_start_range_ns.hrtimer_start.tick_nohz_stop_sched_tick.__tick_nohz_idle_enter
  24972759 ±  2%     -98.3%     417134 ±  9%  time.voluntary_context_switches
      0.94 ± 25%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.hrtimer_cancel.tick_nohz_restart.tick_nohz_idle_exit.cpu_startup_entry.start_secondary
      0.99 ± 19%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.hrtimer_try_to_cancel.hrtimer_cancel.tick_nohz_restart.tick_nohz_idle_exit.cpu_startup_entry
     20.83 ± 15%     -99.3%       0.15 ± 40%  turbostat.CPU%c3
   2456623 ±  8%     -99.2%      20817 ± 38%  sched_debug.cpu#10.ttwu_count
   2493452 ±  9%     -99.2%      20381 ± 38%  sched_debug.cpu#10.sched_goidle
   4988930 ±  9%     -99.2%      42017 ± 37%  sched_debug.cpu#10.sched_count
   4988215 ±  9%     -99.2%      41816 ± 37%  sched_debug.cpu#10.nr_switches
  14655.48 ± 32%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[9]:/.max_vruntime
  14655.48 ± 32%    -100.0%       0.00 ±  0%  sched_debug.cfs_rq[9]:/.MIN_vruntime
   2644593 ±  7%     -99.4%      16074 ± 33%  sched_debug.cpu#9.ttwu_count
   2265014 ± 15%     -99.3%      15600 ± 34%  sched_debug.cpu#9.sched_goidle
   3222460 ±  7%     -98.6%      45873 ± 27%  sched_debug.cpu#0.nr_switches
   4531465 ± 15%     -99.3%      32789 ± 32%  sched_debug.cpu#9.sched_count
   3223037 ±  7%     -98.6%      46060 ± 26%  sched_debug.cpu#0.sched_count
   1609191 ±  7%     -98.7%      20811 ± 29%  sched_debug.cpu#0.sched_goidle
   1498187 ± 10%     -98.4%      23779 ± 27%  sched_debug.cpu#0.ttwu_count
   4530977 ± 15%     -99.3%      32616 ± 32%  sched_debug.cpu#9.nr_switches
   2759516 ±  3%     -99.4%      17615 ± 30%  sched_debug.cpu#8.ttwu_count
   3018619 ±  9%     -99.4%      16773 ± 29%  sched_debug.cpu#8.sched_goidle
   6039570 ±  9%     -99.4%      34816 ± 28%  sched_debug.cpu#8.sched_count
      1.45 ± 11%    -100.0%       0.00 ±  0%  perf-profile.cpu-cycles.dequeue_entity.dequeue_task_fair.dequeue_task.deactivate_task.__sched_text_start
   3261866 ± 12%     -86.4%     443808 ± 24%  sched_debug.cpu#1.nr_switches
   2788959 ±  7%     -99.5%      15010 ± 12%  sched_debug.cpu#7.ttwu_count
   2910769 ±  3%     -99.5%      14317 ± 12%  sched_debug.cpu#7.sched_goidle
   5823925 ±  3%     -99.5%      29602 ± 11%  sched_debug.cpu#7.sched_count
   5822919 ±  3%     -99.5%      29393 ± 11%  sched_debug.cpu#7.nr_switches
   3262414 ± 12%     -86.4%     443998 ± 24%  sched_debug.cpu#1.sched_count
   1630336 ± 12%     -86.4%     221383 ± 24%  sched_debug.cpu#1.sched_goidle
   1590850 ± 11%     -86.1%     221730 ± 24%  sched_debug.cpu#1.ttwu_count
   2244028 ± 15%     -80.0%     448572 ± 20%  sched_debug.cpu#4.sched_count
   1121323 ± 15%     -80.1%     223305 ± 20%  sched_debug.cpu#4.sched_goidle
   2727451 ±  4%     -99.5%      14096 ±  5%  sched_debug.cpu#6.ttwu_count
   1952837 ±  2%     -88.7%     219701 ± 30%  sched_debug.cpu#3.nr_switches
      9443 ± 24%    +510.5%      57651 ± 20%  sched_debug.cfs_rq[5]:/.exec_clock
   5386675 ±  7%     -99.5%      28321 ±  5%  sched_debug.cpu#6.sched_count
   5385896 ±  7%     -99.5%      28079 ±  5%  sched_debug.cpu#6.nr_switches
   2243723 ± 15%     -80.0%     448446 ± 20%  sched_debug.cpu#4.nr_switches
     58333 ± 26%    +800.2%     525096 ± 19%  sched_debug.cfs_rq[5]:/.min_vruntime
   2692398 ±  7%     -99.5%      13778 ±  5%  sched_debug.cpu#6.sched_goidle
     15.29 ±  2%     -83.6%       2.51 ± 34%  perf-profile.cpu-cycles.mutex_unlock.generic_file_write_iter.new_sync_write.vfs_write.sys_write
        12 ± 25%    +431.2%         63 ± 11%  sched_debug.cpu#5.cpu_load[4]
        12 ± 25%    +278.0%         47 ± 29%  sched_debug.cpu#4.cpu_load[4]
        14 ± 20%    +356.1%         65 ± 11%  sched_debug.cpu#5.cpu_load[3]
     88847 ± 19%    +279.8%     337399 ± 37%  sched_debug.cfs_rq[4]:/.min_vruntime
   1031731 ± 16%     -78.4%     223086 ± 20%  sched_debug.cpu#4.ttwu_count
     99804 ± 24%    +338.8%     437987 ± 19%  sched_debug.cfs_rq[3]:/.min_vruntime
      2223 ± 49%    +209.6%       6884 ± 10%  time.involuntary_context_switches
     17.87 ±  5%    +308.0%      72.92 ±  1%  perf-profile.cpu-cycles.__mutex_lock_slowpath.mutex_lock.generic_file_write_iter.new_sync_write.vfs_write
        18 ± 18%    +260.0%         67 ± 12%  sched_debug.cpu#5.cpu_load[2]
      6968 ± 20%    +278.9%      26401 ±  9%  sched_debug.cfs_rq[5]:/.avg->runnable_avg_sum
       151 ± 20%    +279.4%        575 ±  9%  sched_debug.cfs_rq[5]:/.tg_runnable_contrib
    144391 ± 29%    +340.5%     635982 ± 10%  sched_debug.cfs_rq[2]:/.min_vruntime
        13 ± 27%    +250.9%         48 ± 28%  sched_debug.cpu#4.cpu_load[3]
     13600 ± 14%    +165.3%      36078 ± 44%  sched_debug.cfs_rq[4]:/.exec_clock
     14705 ± 16%    +210.9%      45712 ± 23%  sched_debug.cfs_rq[3]:/.exec_clock
     21.26 ±  3%    +249.9%      74.39 ±  1%  perf-profile.cpu-cycles.mutex_lock.generic_file_write_iter.new_sync_write.vfs_write.sys_write
        19 ± 32%    +265.4%         71 ± 12%  sched_debug.cpu#2.cpu_load[4]
    146181 ± 22%    +276.4%     550280 ±  7%  sched_debug.cfs_rq[1]:/.min_vruntime
        14 ± 19%    +250.8%         51 ± 21%  sched_debug.cpu#3.cpu_load[4]
        15 ± 32%    +215.9%         49 ± 27%  sched_debug.cpu#4.cpu_load[2]
     22075 ± 18%    +206.6%      67680 ± 13%  sched_debug.cfs_rq[2]:/.exec_clock
        21 ± 25%    +228.7%         71 ± 12%  sched_debug.cpu#2.cpu_load[3]
     30386 ± 36%    +127.5%      69129 ±  9%  sched_debug.cpu#5.nr_load_updates
    211200 ± 15%    +164.8%     559266 ± 24%  sched_debug.cfs_rq[10]:/.min_vruntime
        26 ± 19%    +170.5%         71 ± 15%  sched_debug.cpu#5.cpu_load[1]
        17 ± 11%    +200.0%         51 ± 21%  sched_debug.cpu#3.cpu_load[3]
        24 ± 24%    +193.8%         71 ± 13%  sched_debug.cpu#2.cpu_load[2]
    115343 ±  5%     -64.3%      41124 ±  5%  softirqs.RCU
      1683 ±  8%     -62.1%        638 ±  6%  cpuidle.POLL.usage
      8658 ± 15%    +144.1%      21138 ± 20%  sched_debug.cfs_rq[4]:/.avg->runnable_avg_sum
       189 ± 15%    +144.4%        462 ± 20%  sched_debug.cfs_rq[4]:/.tg_runnable_contrib
       256 ± 29%    +159.6%        666 ±  8%  sched_debug.cfs_rq[2]:/.tg_runnable_contrib
     11739 ± 29%    +159.9%      30515 ±  8%  sched_debug.cfs_rq[2]:/.avg->runnable_avg_sum
        29 ± 14%    +110.3%         61 ± 21%  sched_debug.cpu#10.cpu_load[4]
    336086 ± 15%    +150.3%     841284 ±  6%  sched_debug.cfs_rq[6]:/.min_vruntime
       969 ± 40%    +109.0%       2025 ±  9%  sched_debug.cpu#1.curr->pid
        21 ±  5%    +145.2%         51 ± 20%  sched_debug.cpu#3.cpu_load[2]
       202 ± 13%    +155.3%        515 ± 12%  sched_debug.cfs_rq[3]:/.tg_runnable_contrib
      9249 ± 13%    +155.4%      23619 ± 12%  sched_debug.cfs_rq[3]:/.avg->runnable_avg_sum
       739 ± 21%    +150.6%       1853 ± 22%  sched_debug.cpu#4.curr->pid
    202942 ± 11%    +125.2%     456974 ± 16%  sched_debug.cfs_rq[9]:/.min_vruntime
       970 ±  6%     -55.7%        430 ± 45%  sched_debug.cpu#6.ttwu_local
        19 ± 37%    +167.9%         52 ± 26%  sched_debug.cpu#4.cpu_load[1]
    452477 ± 19%    +140.5%    1088065 ±  0%  softirqs.TIMER
     22472 ± 11%    +148.8%      55914 ±  7%  sched_debug.cfs_rq[1]:/.exec_clock
        36 ± 27%    +106.9%         75 ± 22%  sched_debug.cfs_rq[5]:/.runnable_load_avg
      1.52 ±  7%     -53.9%       0.70 ±  7%  perf-profile.cpu-cycles.system_call_after_swapgs
        32 ±  9%     +94.5%         62 ± 20%  sched_debug.cpu#10.cpu_load[3]
        21 ± 34%    +183.3%         59 ±  9%  sched_debug.cpu#1.cpu_load[4]
       542 ± 32%     +91.3%       1037 ±  0%  time.system_time
      1.01 ±  8%     -52.1%       0.48 ±  3%  perf-profile.cpu-cycles.__sb_end_write.vfs_write.sys_write.system_call_fastpath
      1.44 ± 16%     -57.3%       0.61 ±  8%  perf-profile.cpu-cycles.sys_lseek.system_call_fastpath
    241726 ± 20%    +161.9%     633158 ±  4%  sched_debug.cfs_rq[7]:/.min_vruntime
        29 ± 26%    +133.6%         69 ± 15%  sched_debug.cpu#2.cpu_load[1]
       186 ± 30%     +86.3%        347 ±  0%  time.percent_of_cpu_this_job_got
        23 ± 15%    +151.1%         57 ± 30%  sched_debug.cfs_rq[4]:/.runnable_load_avg
        46 ± 24%    +144.9%        114 ± 21%  sched_debug.cpu#6.cpu_load[3]
        41 ± 20%     +82.6%         76 ± 28%  sched_debug.cpu#11.cpu_load[0]
       893 ± 11%    +122.5%       1986 ±  4%  sched_debug.cpu#10.curr->pid
        50 ± 28%    +148.0%        124 ± 22%  sched_debug.cpu#6.cpu_load[4]
        47 ± 23%    +131.4%        108 ± 17%  sched_debug.cpu#6.cpu_load[2]
      1.26 ±  7%     -54.5%       0.57 ±  8%  perf-profile.cpu-cycles.__sb_start_write.vfs_write.sys_write.system_call_fastpath
      3.00 ±  8%     -52.7%       1.42 ±  4%  perf-profile.cpu-cycles.shmem_getpage_gfp.shmem_write_begin.generic_perform_write.__generic_file_write_iter.generic_file_write_iter
        31 ± 36%    +130.2%         72 ±  7%  sched_debug.cpu#7.cpu_load[4]
      3.37 ±  7%     -51.4%       1.64 ±  1%  perf-profile.cpu-cycles.shmem_write_begin.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.new_sync_write
        41 ± 27%     +90.2%         78 ± 21%  sched_debug.cpu#5.cpu_load[0]
      1152 ± 36%     +91.5%       2207 ±  9%  sched_debug.cpu#7.curr->pid
     32766 ±  7%     +78.3%      58423 ± 27%  sched_debug.cfs_rq[10]:/.exec_clock
      2.42 ± 13%     -54.2%       1.11 ± 32%  perf-profile.cpu-cycles._raw_spin_lock.__mutex_unlock_slowpath.mutex_unlock.generic_file_write_iter.new_sync_write
        22 ± 32%    +164.4%         59 ±  8%  sched_debug.cpu#1.cpu_load[3]
       998 ± 21%    +108.6%       2081 ±  8%  sched_debug.cpu#8.curr->pid
      1.43 ±  8%     -49.6%       0.72 ± 10%  perf-profile.cpu-cycles.unlock_page.shmem_write_end.generic_perform_write.__generic_file_write_iter.generic_file_write_iter
        27 ±  9%     +85.5%         51 ± 19%  sched_debug.cpu#3.cpu_load[1]
        51 ± 20%    +106.3%        105 ± 12%  sched_debug.cpu#6.cpu_load[1]
      2.20 ±  8%     -51.8%       1.06 ±  6%  perf-profile.cpu-cycles.find_lock_entry.shmem_getpage_gfp.shmem_write_begin.generic_perform_write.__generic_file_write_iter
      7.46 ±  4%     -51.1%       3.65 ±  6%  perf-profile.cpu-cycles.copy_user_generic_string.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.new_sync_write
      2.79 ±  3%     -48.3%       1.44 ±  7%  perf-profile.cpu-cycles.fsnotify.vfs_write.sys_write.system_call_fastpath
       327 ± 17%     +75.1%        573 ± 15%  sched_debug.cfs_rq[10]:/.tg_runnable_contrib
     14967 ± 17%     +75.2%      26221 ± 15%  sched_debug.cfs_rq[10]:/.avg->runnable_avg_sum
         2 ± 15%    +109.1%          5 ±  7%  vmstat.procs.r
        24 ± 25%    +145.5%         60 ±  6%  sched_debug.cpu#1.cpu_load[1]
        30 ± 21%     +99.2%         59 ± 13%  sched_debug.cpu#8.cpu_load[4]
        23 ± 29%    +151.6%         59 ±  7%  sched_debug.cpu#1.cpu_load[2]
     19.91 ±  3%     -48.4%      10.27 ±  3%  perf-profile.cpu-cycles.__generic_file_write_iter.generic_file_write_iter.new_sync_write.vfs_write.sys_write
     17.33 ±  4%     -48.1%       8.99 ±  3%  perf-profile.cpu-cycles.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.new_sync_write.vfs_write
      3.35 ±  2%     -46.9%       1.78 ± 16%  perf-profile.cpu-cycles.shmem_write_end.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.new_sync_write
        29 ± 29%     +85.7%         55 ± 17%  sched_debug.cpu#9.cpu_load[4]
      1.18 ±  9%     -51.8%       0.57 ± 11%  perf-profile.cpu-cycles.find_get_entry.find_lock_entry.shmem_getpage_gfp.shmem_write_begin.generic_perform_write
      1.31 ±  5%     -51.0%       0.65 ±  7%  perf-profile.cpu-cycles.system_call
     12371 ± 18%    +118.5%      27032 ±  6%  sched_debug.cfs_rq[1]:/.avg->runnable_avg_sum
    244456 ± 15%    +123.7%     546812 ± 14%  sched_debug.cfs_rq[8]:/.min_vruntime
       271 ± 18%    +118.0%        590 ±  6%  sched_debug.cfs_rq[1]:/.tg_runnable_contrib
        36 ±  9%     +78.5%         64 ± 18%  sched_debug.cpu#10.cpu_load[2]
        35 ± 38%    +103.5%         72 ±  6%  sched_debug.cpu#7.cpu_load[3]
        31 ± 19%     +90.5%         60 ± 13%  sched_debug.cpu#8.cpu_load[3]
      3694 ± 15%     +94.6%       7190 ±  0%  sched_debug.cfs_rq[0]:/.tg->runnable_avg
      3694 ± 15%     +94.4%       7183 ±  0%  sched_debug.cfs_rq[1]:/.tg->runnable_avg
        37 ± 22%     +73.8%         64 ±  5%  sched_debug.cfs_rq[1]:/.runnable_load_avg
      3695 ± 15%     +93.8%       7161 ±  0%  sched_debug.cfs_rq[2]:/.tg->runnable_avg
      3696 ± 15%     +93.7%       7162 ±  0%  sched_debug.cfs_rq[3]:/.tg->runnable_avg
      3701 ± 15%     +93.6%       7165 ±  0%  sched_debug.cfs_rq[5]:/.tg->runnable_avg
      3698 ± 15%     +93.7%       7162 ±  0%  sched_debug.cfs_rq[4]:/.tg->runnable_avg
      3703 ± 15%     +93.5%       7166 ±  0%  sched_debug.cfs_rq[6]:/.tg->runnable_avg
      3711 ± 15%     +93.3%       7173 ±  0%  sched_debug.cfs_rq[9]:/.tg->runnable_avg
      3711 ± 15%     +93.2%       7169 ±  0%  sched_debug.cfs_rq[8]:/.tg->runnable_avg
      3707 ± 15%     +93.3%       7167 ±  0%  sched_debug.cfs_rq[7]:/.tg->runnable_avg
      3714 ± 15%     +93.1%       7174 ±  0%  sched_debug.cfs_rq[10]:/.tg->runnable_avg
      3717 ± 15%     +93.1%       7177 ±  0%  sched_debug.cfs_rq[11]:/.tg->runnable_avg
       457 ± 16%     +80.3%        824 ±  4%  sched_debug.cfs_rq[6]:/.tg_runnable_contrib
       327 ± 19%     +63.5%        535 ± 15%  sched_debug.cfs_rq[9]:/.tg_runnable_contrib
      5.21 ±  8%     +94.9%      10.14 ±  1%  perf-profile.cpu-cycles.mutex_spin_on_owner.mutex_optimistic_spin.__mutex_lock_slowpath.mutex_lock.generic_file_write_iter
     20973 ± 16%     +80.4%      37832 ±  4%  sched_debug.cfs_rq[6]:/.avg->runnable_avg_sum
     14981 ± 19%     +63.9%      24548 ± 15%  sched_debug.cfs_rq[9]:/.avg->runnable_avg_sum
        36 ± 20%     +38.8%         51 ± 19%  sched_debug.cpu#3.cpu_load[0]
      1052 ± 22%     +68.3%       1770 ± 15%  sched_debug.cpu#3.curr->pid
        41 ± 42%     +78.7%         73 ±  5%  sched_debug.cpu#7.cpu_load[2]
        42 ± 33%     +63.9%         69 ± 19%  sched_debug.cfs_rq[2]:/.runnable_load_avg
        28 ± 25%    +121.4%         62 ±  5%  sched_debug.cpu#1.cpu_load[0]
      2973 ±  6%     -41.9%       1727 ±  0%  uptime.idle
      1370 ± 10%     +68.7%       2311 ±  9%  sched_debug.cpu#6.curr->pid
    301194 ±  6%     +73.9%     523791 ± 10%  sched_debug.cpu#11.avg_idle
        32 ± 14%     +83.8%         59 ± 13%  sched_debug.cpu#8.cpu_load[2]
       234 ± 30%     +45.9%        341 ± 13%  sched_debug.cfs_rq[2]:/.tg_load_contrib
        41 ± 17%     +64.1%         68 ± 17%  sched_debug.cpu#10.cpu_load[1]
     61684 ± 20%     +85.4%     114380 ±  9%  sched_debug.cfs_rq[6]:/.exec_clock
    315489 ± 24%     +83.4%     578759 ± 10%  sched_debug.cfs_rq[0]:/.min_vruntime
 1.474e+08 ±  8%     -42.1%   85363914 ±  6%  cpuidle.C1E-NHM.time
      1.15 ±  4%     -39.4%       0.70 ± 12%  perf-profile.cpu-cycles.__srcu_read_lock.fsnotify.vfs_write.sys_write.system_call_fastpath
        28 ± 40%    +105.4%         57 ± 27%  sched_debug.cpu#4.cpu_load[0]
        35 ± 14%     +68.8%         59 ± 13%  sched_debug.cpu#8.cpu_load[1]
        31 ± 12%     +54.0%         47 ± 13%  sched_debug.cpu#11.cpu_load[4]
        59 ± 22%     +75.8%        103 ± 14%  sched_debug.cpu#6.cpu_load[0]
      1137 ± 25%     +92.1%       2184 ± 14%  sched_debug.cpu#5.curr->pid
        32 ± 27%     +72.5%         56 ± 16%  sched_debug.cpu#9.cpu_load[3]
    264073 ±  5%     +49.3%     394342 ± 13%  sched_debug.cpu#8.avg_idle
     37082 ± 12%     +79.2%      66466 ±  6%  sched_debug.cfs_rq[7]:/.exec_clock
        39 ± 22%     +46.8%         58 ± 14%  sched_debug.cpu#8.cpu_load[0]
     31977 ±  5%     +51.3%      48388 ± 22%  sched_debug.cfs_rq[9]:/.exec_clock
     17401 ± 15%     +79.1%      31162 ±  4%  sched_debug.cfs_rq[7]:/.avg->runnable_avg_sum
       380 ± 15%     +78.6%        679 ±  4%  sched_debug.cfs_rq[7]:/.tg_runnable_contrib
        40 ± 13%     +72.4%         70 ± 17%  sched_debug.cfs_rq[10]:/.runnable_load_avg
        37 ± 27%     +58.8%         58 ±  9%  sched_debug.cfs_rq[8]:/.runnable_load_avg
  38761708 ± 33%     +61.8%   62732448 ±  7%  cpuidle.POLL.time
      1178 ± 19%     +83.1%       2158 ±  5%  sched_debug.cpu#0.curr->pid
      1220 ±  4%     +55.6%       1899 ± 11%  sched_debug.cpu#11.curr->pid
     57975 ±  9%     +40.6%      81536 ±  8%  sched_debug.cpu#2.nr_load_updates
    166493 ± 16%     -41.6%      97254 ± 35%  sched_debug.cpu#3.ttwu_local
     75051 ±  6%     -45.2%      41162 ± 24%  sched_debug.cpu#11.nr_load_updates
     57.44 ±  2%     +52.6%      87.64 ±  1%  perf-profile.cpu-cycles.generic_file_write_iter.new_sync_write.vfs_write.sys_write.system_call_fastpath
        35 ± 13%     +57.4%         55 ± 10%  sched_debug.cpu#11.cpu_load[2]
    293147 ±  5%     +41.2%     413910 ± 19%  sched_debug.cpu#10.avg_idle
        32 ± 12%     +55.7%         51 ±  9%  sched_debug.cpu#11.cpu_load[3]
        38 ± 16%     +66.7%         63 ± 18%  sched_debug.cpu#11.cpu_load[1]
     16272 ± 14%     +32.4%      21550 ± 13%  sched_debug.cfs_rq[11]:/.avg->runnable_avg_sum
       356 ± 14%     +32.1%        471 ± 13%  sched_debug.cfs_rq[11]:/.tg_runnable_contrib
      2315 ±  1%     +48.7%       3444 ±  1%  proc-vmstat.pgactivate
     61.08 ±  2%     +47.4%      90.04 ±  2%  perf-profile.cpu-cycles.new_sync_write.vfs_write.sys_write.system_call_fastpath
        46 ± 26%     +66.3%         76 ± 18%  sched_debug.cpu#10.cpu_load[0]
    395145 ± 15%     -28.1%     283954 ± 15%  sched_debug.cpu#2.avg_idle
     16930 ± 13%     +62.2%      27468 ±  8%  sched_debug.cfs_rq[8]:/.avg->runnable_avg_sum
       370 ± 13%     +62.0%        600 ±  8%  sched_debug.cfs_rq[8]:/.tg_runnable_contrib
      1111 ± 35%     +81.5%       2016 ± 16%  sched_debug.cpu#2.curr->pid
        53 ± 24%     +72.0%         92 ± 13%  sched_debug.cfs_rq[6]:/.runnable_load_avg
      1264 ± 16%     +50.8%       1906 ±  1%  sched_debug.cpu#9.curr->pid
     76679 ±  3%     -26.1%      56701 ± 15%  sched_debug.cpu#8.nr_load_updates
     32.91 ±  5%     -26.8%      24.09 ±  1%  turbostat.CPU%c1
     18589 ± 18%     +64.6%      30603 ±  8%  sched_debug.cfs_rq[0]:/.avg->runnable_avg_sum
       405 ± 18%     +64.4%        667 ±  8%  sched_debug.cfs_rq[0]:/.tg_runnable_contrib
     37668 ±  8%     +44.7%      54510 ± 16%  sched_debug.cfs_rq[8]:/.exec_clock
     68.80 ±  2%     +36.2%      93.71 ±  2%  perf-profile.cpu-cycles.vfs_write.sys_write.system_call_fastpath
     70.12 ±  2%     +34.5%      94.33 ±  1%  perf-profile.cpu-cycles.sys_write.system_call_fastpath
     26.11 ±  5%     -22.7%      20.18 ±  2%  time.user_time
     71.98 ±  2%     +32.2%      95.16 ±  1%  perf-profile.cpu-cycles.system_call_fastpath
   1191441 ±  3%     -22.4%     924579 ±  0%  cpuidle.C1E-NHM.usage
      2.82 ±  5%     -27.8%       2.04 ± 12%  perf-profile.cpu-cycles.mutex_unlock.new_sync_write.vfs_write.sys_write.system_call_fastpath
      2741 ±  0%     +27.5%       3496 ±  0%  proc-vmstat.nr_shmem
     10969 ±  0%     +27.5%      13987 ±  0%  meminfo.Shmem
     67072 ±  3%     -25.0%      50333 ± 21%  sched_debug.cpu#9.nr_load_updates
    253442 ±  8%     +29.3%     327656 ± 10%  sched_debug.cpu#7.avg_idle
        61 ±  6%     +13.9%         69 ±  1%  turbostat.CoreTmp
       121 ±  6%     -18.5%         99 ± 23%  sched_debug.cfs_rq[0]:/.load
     76604 ±  3%     -11.6%      67691 ±  6%  sched_debug.cpu#7.nr_load_updates
     23935 ±  3%     +12.8%      26987 ±  2%  meminfo.Active(anon)
      5982 ±  3%     +12.8%       6745 ±  2%  proc-vmstat.nr_active_anon
     63339 ± 10%     +13.7%      72028 ±  4%  sched_debug.cpu#1.nr_load_updates
     69405 ±  3%      -7.2%      64386 ±  4%  meminfo.DirectMap4k
    401667 ±  2%     -97.0%      11907 ±  3%  vmstat.system.cs
     29.02 ± 16%    +106.1%      59.81 ±  0%  turbostat.%Busy
      1022 ± 16%    +105.8%       2103 ±  0%  turbostat.Avg_MHz
      7739 ±  9%     +34.9%      10437 ±  0%  vmstat.system.in

wsm: Westmere
Memory: 6G




                            time.voluntary_context_switches

    3e+07 ++------------------------*--*------------------------------------+
          *..                      :    +                 *                 |
  2.5e+07 ++     .*..*.*..         :     +    .*..*.*.. .. +  .*.     .*.*..*
          |  *.*.         *.*..*..*       *.*.         *    *.   *..*.      |
          |                                                                 |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++                                                                |
          |                                                                 |
    1e+07 ++                                                                |
          |                                                                 |
          |                                                                 |
    5e+06 ++                                                                |
          |                                                                 |
        0 O+-O-O--O--O-O--O-O--O--O-O--O--O-O--O--O-O--O--O-O--O------------+


                                   softirqs.SCHED

  450000 ++-----------------------------------------------------------------+
         |            *..                                              *    |
  400000 ++          :     .*                                         : :   *
  350000 ++          :   *.  +                           *..          : : ..|
         |   .*..*..*         *..*..       .*..*.       +            :   *  |
  300000 *+.*                       *.*..*.      *..*..*    *..*.*..*       |
  250000 ++                                                                 |
         |                                                                  |
  200000 ++                                                                 |
  150000 ++                                                                 |
         |                                                                  |
  100000 ++                                                                 |
   50000 ++                                                                 |
         O  O O  O  O O  O  O O  O  O O  O  O  O O  O  O O  O  O            |
       0 ++-----------------------------------------------------------------+


                                  softirqs.HRTIMER

  12000 ++------------------------------------------------------------------+
        |                               .*.*..*..*                          |
  10000 ++                    *.  .*..*.          :                         |
        | .*                ..  *.                :   *..        *..*..     |
        *.  +  .*..*..     *                       : :   *..   ..      *.*..*
   8000 ++   *.           +                        : :      *.*             |
        |             *. +                          *                       |
   6000 ++              *                                                   |
        |                                                                   |
   4000 ++                                                                  |
        |                                                                   |
        |                                                                   |
   2000 ++                                                                  |
        |                                                                   |
      0 O+-O-O--O--O--O-O--O--O-O--O--O--O-O--O--O--O-O--O--O-O-------------+


                                    softirqs.RCU

  130000 ++-----------------------------------------------------------------+
  120000 ++                                                              *..|
         | .*.  .*..*       *.*..*..*.  .*..*..*.*..*..          *..*.. +   *
  110000 *+   *.     +     +          *.               *.*..    +      *    |
  100000 ++           *.. +                                 *..*            |
         |               *                                                  |
   90000 ++                                                                 |
   80000 ++                                                                 |
   70000 ++                                                                 |
         |                                                                  |
   60000 ++                                                                 |
   50000 ++                                                                 |
         |                    O                             O  O            |
   40000 O+ O O  O  O O  O  O    O  O O  O  O  O O  O  O O                  |
   30000 ++-----------------------------------------------------------------+


                     will-it-scale.time.voluntary_context_switches

    3e+07 ++------------------------*--*------------------------------------+
          *..                      :    +                 *                 |
  2.5e+07 ++     .*..*.*..         :     +    .*..*.*.. .. +  .*.     .*.*..*
          |  *.*.         *.*..*..*       *.*.         *    *.   *..*.      |
          |                                                                 |
    2e+07 ++                                                                |
          |                                                                 |
  1.5e+07 ++                                                                |
          |                                                                 |
    1e+07 ++                                                                |
          |                                                                 |
          |                                                                 |
    5e+06 ++                                                                |
          |                                                                 |
        0 O+-O-O--O--O-O--O-O--O--O-O--O--O-O--O--O-O--O--O-O--O------------+


                                  vmstat.system.cs

  450000 ++-----------------------------------------------------------------+
         *..                       .*.*..                        *..  .*.*..|
  400000 ++ *    *..*       *.*..*.      *..*..*.*..   *.*..    :   *.      *
  350000 ++  + ..    +     :                         ..     *.. :           |
         |    *       *..  :                        *          *            |
  300000 ++               :                                                 |
  250000 ++              *                                                  |
         |                                                                  |
  200000 ++                                                                 |
  150000 ++                                                                 |
         |                                                                  |
  100000 ++                                                                 |
   50000 ++                                                                 |
         |                                                                  |
       0 O+-O-O--O--O-O--O--O-O--O--O-O--O--O--O-O--O--O-O--O--O------------+


                             sched_debug.cpu#0.nr_switches

    4e+06 ++----------------------------------------------------------------+
          |                                                            *    |
  3.5e+06 ++                                                         .. +   |
    3e+06 *+.                       *..                          *..*    *..|
          |  *    *..*.       .*.. +   *..*.*..*..*.  .*..*.*.. +           *
  2.5e+06 ++  + ..     *..  *.    *                 *.         *            |
          |    *           +                                                |
    2e+06 ++              *                                                 |
          |                                                                 |
  1.5e+06 ++                                                                |
    1e+06 ++                                                                |
          |                                                                 |
   500000 ++                                                                |
          |                                                                 |
        0 O+-O-O--O--O-O--O-O--O--O-O--O--O-O--O--O-O--O--O-O--O------------+


                             sched_debug.cpu#0.sched_count

    4e+06 ++----------------------------------------------------------------+
          |                                                            *    |
  3.5e+06 ++                                                         .. +   |
    3e+06 *+.                       *..                          *..*    *..|
          |  *    *..*.       .*.. +   *..*.*..*..*.  .*..*.*.. +           *
  2.5e+06 ++  + ..     *..  *.    *                 *.         *            |
          |    *           +                                                |
    2e+06 ++              *                                                 |
          |                                                                 |
  1.5e+06 ++                                                                |
    1e+06 ++                                                                |
          |                                                                 |
   500000 ++                                                                |
          |                                                                 |
        0 O+-O-O--O--O-O--O-O--O--O-O--O--O-O--O--O-O--O--O-O--O------------+


                            sched_debug.cpu#0.sched_goidle

  1.8e+06 ++-----------------------------------------------------------*----+
          |                                                          ..  *..|
  1.6e+06 *+.                       *..*..                       *..*       |
  1.4e+06 ++ *    *..*.       .*.. +      *.*..*..*.  .*..*.*.. +           *
          |   + ..     *..  *.    *                 *.         *            |
  1.2e+06 ++   *           +                                                |
    1e+06 ++              *                                                 |
          |                                                                 |
   800000 ++                                                                |
   600000 ++                                                                |
          |                                                                 |
   400000 ++                                                                |
   200000 ++                                                                |
          |                                                                 |
        0 O+-O-O--O--O-O--O-O--O--O-O--O--O-O--O--O-O--O--O-O--O------------+


                             sched_debug.cpu#0.ttwu_count

  1.8e+06 ++----------------------------------------------------------------+
          |                                               *            *.   |
  1.6e+06 ++                        *..                  : +         ..  *..|
  1.4e+06 *+.     *..         .*.. +   *.. .*..*..       :  *..  *..*       |
          |  *   +   *.*..  *.    *       *       *.*.. :       +           *
  1.2e+06 ++  + +          +                           *       *            |
    1e+06 ++   *          *                                                 |
          |                                                                 |
   800000 ++                                                                |
   600000 ++                                                                |
          |                                                                 |
   400000 ++                                                                |
   200000 ++                                                                |
          |                                                                 |
        0 O+-O-O--O--O-O--O-O--O--O-O--O--O-O--O--O-O--O--O-O--O------------+


                            sched_debug.cpu#6.nr_switches

  7e+06 ++------------------------------------------------------------------+
        |                                                                   |
  6e+06 ++         *                                             *..*..     |
        | .*      + :             .*..*..            .*         :      *.   |
  5e+06 *+  :    +  :      *..*.*.       *.*..*..*..*  +        :        *..*
        |   :  .*    :   ..                             +      :            |
  4e+06 ++   *.      : .*                                *..*. :            |
        |             *                                       *             |
  3e+06 ++                                                                  |
        |                                                                   |
  2e+06 ++                                                                  |
        |                                                                   |
  1e+06 ++                                                                  |
        |                                                                   |
      0 O+-O-O--O--O--O-O--O--O-O--O--O--O-O--O--O--O-O--O--O-O-------------+


                            sched_debug.cpu#6.sched_count

  7e+06 ++------------------------------------------------------------------+
        |                                                                   |
  6e+06 ++         *                                             *..*..     |
        | .*      + :             .*..*..            .*         :      *.   |
  5e+06 *+  :    +  :      *..*.*.       *.*..*..*..*  +        :        *..*
        |   :  .*    :   ..                             +      :            |
  4e+06 ++   *.      : .*                                *..*. :            |
        |             *                                       *             |
  3e+06 ++                                                                  |
        |                                                                   |
  2e+06 ++                                                                  |
        |                                                                   |
  1e+06 ++                                                                  |
        |                                                                   |
      0 O+-O-O--O--O--O-O--O--O-O--O--O--O-O--O--O--O-O--O--O-O-------------+


                            sched_debug.cpu#6.sched_goidle

  3.5e+06 ++----------------------------------------------------------------+
          |                                                                 |
    3e+06 ++         *                                           *..*..     |
          | .*      +:             .*..*..            .*         :     *.   |
  2.5e+06 *+  :    +  :     *..*..*       *.*..*..*.*.  +       :        *..*
          |   :  .*   :    +                             +      :           |
    2e+06 ++   *.      : .*                               *.*..:            |
          |            *.                                      *            |
  1.5e+06 ++                                                                |
          |                                                                 |
    1e+06 ++                                                                |
          |                                                                 |
   500000 ++                                                                |
          |                                                                 |
        0 O+-O-O--O--O-O--O-O--O--O-O--O--O-O--O--O-O--O--O-O--O------------+


                             sched_debug.cpu#6.ttwu_count

    3e+06 ++----------------------------------------------------------------+
          |         .*           .*.             .*.             *..  .*.*..*
  2.5e+06 *+.     *.  :     *..*.   *..*..*.*..*.   *..*..       :  *.      |
          |  *   +    :    :                              *.    :           |
          |   + +      :   :                                *.. :           |
    2e+06 ++   *       *..*                                    :            |
          |                                                    *            |
  1.5e+06 ++                                                                |
          |                                                                 |
    1e+06 ++                                                                |
          |                                                                 |
          |                                                                 |
   500000 ++                                                                |
          |                                                                 |
        0 O+-O-O--O--O-O--O-O--O--O-O--O--O-O--O--O-O--O--O-O--O------------+


                            sched_debug.cpu#7.nr_switches

  7e+06 ++------------------------------------------------------------------+
        |                     *                                             |
  6e+06 ++                  .. +     .*..                          .*..    .*
        *..*    *..*       *    *..*.    *.*..*..*..*.  .*..     *.    *.*. |
  5e+06 ++  :  +    +     +                           *.    *. ..           |
        |   : +      +   +                                    *             |
  4e+06 ++   *        *.*                                                   |
        |                                                                   |
  3e+06 ++                                                                  |
        |                                                                   |
  2e+06 ++                                                                  |
        |                                                                   |
  1e+06 ++                                                                  |
        |                                                                   |
      0 O+-O-O--O--O--O-O--O--O-O--O--O--O-O--O--O--O-O--O--O-O-------------+


                            sched_debug.cpu#7.sched_count

  7e+06 ++------------------------------------------------------------------+
        |                     *                                             |
  6e+06 ++                  .. +     .*..                          .*..    .*
        *..*    *..*       *    *..*.    *.*..*..*..*.  .*..     *.    *.*. |
  5e+06 ++  :  +    +     +                           *.    *. ..           |
        |   : +      +   +                                    *             |
  4e+06 ++   *        *.*                                                   |
        |                                                                   |
  3e+06 ++                                                                  |
        |                                                                   |
  2e+06 ++                                                                  |
        |                                                                   |
  1e+06 ++                                                                  |
        |                                                                   |
      0 O+-O-O--O--O--O-O--O--O-O--O--O--O-O--O--O--O-O--O--O-O-------------+


                            sched_debug.cpu#7.sched_goidle

  3.5e+06 ++----------------------------------------------------------------+
          |                    *..                                          |
    3e+06 ++                 ..       .*..                         .*..    .*
          *..*    *..*      *     *.*.    *.*..*..*.*..  .*.     *.    *.*. |
  2.5e+06 ++  :  +    :    :                           *.   *.. +           |
          |   : +     :    :                                   *            |
    2e+06 ++   *       *..*                                                 |
          |                                                                 |
  1.5e+06 ++                                                                |
          |                                                                 |
    1e+06 ++                                                                |
          |                                                                 |
   500000 ++                                                                |
          |                                                                 |
        0 O+-O-O--O--O-O--O-O--O--O-O--O--O-O--O--O-O--O--O-O--O------------+


                             sched_debug.cpu#7.ttwu_count

  3.5e+06 ++----------------------------------------------------------------+
          |                                                                 |
    3e+06 ++.*                     .*     *.                           *.  .*
          *.  :   *..*           .*  +   +  *..    .*..              ..  *. |
  2.5e+06 ++  :  +    :     *..*.     + +      *..*    *..*     .*..*       |
          |    :+     :    +           *                   +  .*            |
    2e+06 ++   *       *..*                                 *.              |
          |                                                                 |
  1.5e+06 ++                                                                |
          |                                                                 |
    1e+06 ++                                                                |
          |                                                                 |
   500000 ++                                                                |
          |                                                                 |
        0 O+-O-O--O--O-O--O-O--O--O-O--O--O-O--O--O-O--O--O-O--O------------+


                            sched_debug.cpu#8.nr_switches

  7e+06 ++------------------------------------------------------------------+
        |       *                        *                         .*    *..|
  6e+06 ++     : +                .*.. .. +  .*..     *..        *.  +  :   *
        *..    :  +          .*.*.    *    *.    *.. +   *     ..     + :   |
  5e+06 ++ *  :    *..    .*.                       *     +   *        *    |
        |   + :        .*.                                 + +              |
  4e+06 ++   *        *                                     *               |
        |                                                                   |
  3e+06 ++                                                                  |
        |                                                                   |
  2e+06 ++                                                                  |
        |                                                                   |
  1e+06 ++                                                                  |
        |                                                                   |
      0 O+-O-O--O--O--O-O--O--O-O--O--O--O-O--O--O--O-O--O--O-O-------------+


                            sched_debug.cpu#8.sched_count

  7e+06 ++------------------------------------------------------------------+
        |       *                        *                         .*    *..|
  6e+06 ++     : +                .*.. .. +  .*..     *..        *.  +  :   *
        *..    :  +          .*.*.    *    *.    *.. +   *     ..     + :   |
  5e+06 ++ *  :    *..    .*.                       *     +   *        *    |
        |   + :        .*.                                 + +              |
  4e+06 ++   *        *                                     *               |
        |                                                                   |
  3e+06 ++                                                                  |
        |                                                                   |
  2e+06 ++                                                                  |
        |                                                                   |
  1e+06 ++                                                                  |
        |                                                                   |
      0 O+-O-O--O--O--O-O--O--O-O--O--O--O-O--O--O--O-O--O--O-O-------------+


                            sched_debug.cpu#8.sched_goidle

  3.5e+06 ++----------------------------------------------------------------+
          |       *                       *                        .*    *..|
    3e+06 ++     : +               .*.. .. +  .*..     *..       *.  +  :   *
          *..    :  +         .*..*    *    *.    *. ..   *     +     + :   |
  2.5e+06 ++ *  :    *     .*.                      *      :   *       *    |
          |   + :     +  .*                                : ..             |
    2e+06 ++   *       *.                                   *               |
          |                                                                 |
  1.5e+06 ++                                                                |
          |                                                                 |
    1e+06 ++                                                                |
          |                                                                 |
   500000 ++                                                                |
          |                                                                 |
        0 O+-O-O--O--O-O--O-O--O--O-O--O--O-O--O--O-O--O--O-O--O------------+


                             sched_debug.cpu#8.ttwu_count

  3.5e+06 ++----------------------------------------------------------------+
          |                                                                 |
    3e+06 ++                          .*..                       *..        |
          *..                  *..*.*.     .*..*..*.            :   *..*.*..*
  2.5e+06 ++      *..        ..           *         *..  .*     :           |
          |  *. ..   *.  .*.*                          *.  :   *            |
    2e+06 ++   *       *.                                  : ..             |
          |                                                 *               |
  1.5e+06 ++                                                                |
          |                                                                 |
    1e+06 ++                                                                |
          |                                                                 |
   500000 ++                                                                |
          |                                                                 |
        0 O+-O-O--O--O-O--O-O--O--O-O--O--O-O--O--O-O--O--O-O--O------------+


                            sched_debug.cpu#9.nr_switches

  7e+06 ++------------------------------------------------------------------+
        |                                                                   |
  6e+06 *+                      *..                      *       *          |
        |+         *           +   *..                  : :     : :    *    |
  5e+06 +++      .. :         *                         : :     :  :  + :   *
        |  *.  .*   :   *.. ..        *..*.  .*..*..*. :   :   :   : +  : ..|
  4e+06 ++   *.      : :   *               *.         *    :   :    *    *  |
        |            : :                                    :.*             |
  3e+06 ++            *                                     *               |
        |                                                                   |
  2e+06 ++                                                                  |
        |                                                                   |
  1e+06 ++                                                                  |
        |                                                                   |
      0 O+-O-O--O--O--O-O--O--O-O--O--O--O-O--O--O--O-O--O--O-O-------------+


                            sched_debug.cpu#9.sched_count

  7e+06 ++------------------------------------------------------------------+
        |                                                                   |
  6e+06 *+                      *..                      *       *          |
        |+         *           +   *..                  : :     : :    *    |
  5e+06 +++      .. :         *                         : :     :  :  + :   *
        |  *.  .*   :   *.. ..        *..*.  .*..*..*. :   :   :   : +  : ..|
  4e+06 ++   *.      : :   *               *.         *    :   :    *    *  |
        |            : :                                    :.*             |
  3e+06 ++            *                                     *               |
        |                                                                   |
  2e+06 ++                                                                  |
        |                                                                   |
  1e+06 ++                                                                  |
        |                                                                   |
      0 O+-O-O--O--O--O-O--O--O-O--O--O--O-O--O--O--O-O--O--O-O-------------+


                            sched_debug.cpu#9.sched_goidle

  3.5e+06 ++----------------------------------------------------------------+
          |                                                                 |
    3e+06 *+                      *.                      *      *          |
          |+         *          ..  *..                  ::      ::    *    |
  2.5e+06 +++      ..:         *                         : :    :  :  + :   *
          |  *.  .*   :   *. ..        *..*.  .*..*.*.. :  :    :  : +  : ..|
    2e+06 ++   *.     :  +  *               *.         *   :   :    *    *  |
          |            :+                                   : .*            |
  1.5e+06 ++           *                                    *.              |
          |                                                                 |
    1e+06 ++                                                                |
          |                                                                 |
   500000 ++                                                                |
          |                                                                 |
        0 O+-O-O--O--O-O--O-O--O--O-O--O--O-O--O--O-O--O--O-O--O------------+


                             sched_debug.cpu#9.ttwu_count

  3.5e+06 ++----------------------------------------------------------------+
          |                                                                 |
    3e+06 ++                        *..        *..               *..*..     |
          *..*      .*             +   *..*   +   *              :     *.*..|
  2.5e+06 ++  :   *. :      *..  .*        + +     +  .*..*     :           |
          |   : ..    :    +   *.           *       *.     +    :           *
    2e+06 ++   *      :   *                                 *..*            |
          |            :..                                                  |
  1.5e+06 ++           *                                                    |
          |                                                                 |
    1e+06 ++                                                                |
          |                                                                 |
   500000 ++                                                                |
          |                                                                 |
        0 O+-O-O--O--O-O--O-O--O--O-O--O--O-O--O--O-O--O--O-O--O------------+


                           sched_debug.cpu#10.nr_switches

  7e+06 ++------------------------------------------------------------------+
        |                                                                   |
  6e+06 ++                      *..                                         |
        *..                    :   *..                           *..  .*.*  |
  5e+06 ++        .*       *.. :        .*.*..*..               +   *.    + |
        |  *.  .*.  :    ..   *       *.         *..*.         +           +|
  4e+06 ++   *.      :  *                             *..*..*.*             *
        |            : +                                                    |
  3e+06 ++            *                                                     |
        |                                                                   |
  2e+06 ++                                                                  |
        |                                                                   |
  1e+06 ++                                                                  |
        |                                                                   |
      0 O+-O-O--O--O--O-O--O--O-O--O--O--O-O--O--O--O-O--O--O-O-------------+


                           sched_debug.cpu#10.sched_count

  7e+06 ++------------------------------------------------------------------+
        |                                                                   |
  6e+06 ++                      *..                                         |
        *..                    :   *..                           *..  .*.*  |
  5e+06 ++        .*       *.. :        .*.*..*..               +   *.    + |
        |  *.  .*.  :    ..   *       *.         *..*.         +           +|
  4e+06 ++   *.      :  *                             *..*..*.*             *
        |            : +                                                    |
  3e+06 ++            *                                                     |
        |                                                                   |
  2e+06 ++                                                                  |
        |                                                                   |
  1e+06 ++                                                                  |
        |                                                                   |
      0 O+-O-O--O--O--O-O--O--O-O--O--O--O-O--O--O--O-O--O--O-O-------------+


                            sched_debug.cpu#10.sched_goidle

  3.5e+06 ++----------------------------------------------------------------+
          |                                                                 |
    3e+06 ++                      *.                                        |
          *..                    +  *..                          *..  .*.*  |
  2.5e+06 ++        .*      *.. +        .*.*..*..              :   *.    + |
          |  *.  .*.  :    +   *       *.         *.*..         :          +|
    2e+06 ++   *.     :   *                            *..*.*..*            *
          |            :..                                                  |
  1.5e+06 ++           *                                                    |
          |                                                                 |
    1e+06 ++                                                                |
          |                                                                 |
   500000 ++                                                                |
          |                                                                 |
        0 O+-O-O--O--O-O--O-O--O--O-O--O--O-O--O--O-O--O--O-O--O------------+


                             sched_debug.cpu#10.ttwu_count

    3e+06 ++-*---------------------------------------------------*----------+
          *. :                     .*..    .*..                  :  *..     |
  2.5e+06 ++  :               .*..*    *..*    *..*.            :      *.   |
          |   :            .*.                      *..*..*.    :        *..|
          |    : .*..*    *                                 *..:            *
    2e+06 ++   *.     :  +                                     *            |
          |           : +                                                   |
  1.5e+06 ++           *                                                    |
          |                                                                 |
    1e+06 ++                                                                |
          |                                                                 |
          |                                                                 |
   500000 ++                                                                |
          |                                                                 |
        0 O+-O-O--O--O-O--O-O--O--O-O--O--O-O--O--O-O--O--O-O--O------------+


                           sched_debug.cpu#11.nr_switches

  7e+06 ++------------------------------------------------------------------+
        |                                 .*..  .*..                        |
  6e+06 *+.*                       *..  .*    *.                 *..  .*    |
        |  :                      +   *.            *.*..       +   *.  :   |
  5e+06 ++  :     .*             +                       *     +        :  .*
        |   :   *.  +   *..*..*.*                         :   *          *. |
  4e+06 ++  :  +     + +                                   : :              |
        |    :+       *                                    : :              |
  3e+06 ++   *                                              *               |
        |                                                                   |
  2e+06 ++                                                                  |
        |                                                                   |
  1e+06 ++                                                                  |
        |                                                                   |
      0 O+-O-O--O--O--O-O--O--O-O--O--O--O-O--O--O--O-O--O--O-O-------------+


                           sched_debug.cpu#11.sched_count

  7e+06 ++------------------------------------------------------------------+
        |                                 .*..  .*..                        |
  6e+06 *+.*                       *..  .*    *.                 *..  .*    |
        |  :                      +   *.            *.*..       +   *.  :   |
  5e+06 ++  :     .*             +                       *     +        :  .*
        |   :   *.  +   *..*..*.*                         :   *          *. |
  4e+06 ++  :  +     + +                                   : :              |
        |    :+       *                                    : :              |
  3e+06 ++   *                                              *               |
        |                                                                   |
  2e+06 ++                                                                  |
        |                                                                   |
  1e+06 ++                                                                  |
        |                                                                   |
      0 O+-O-O--O--O--O-O--O--O-O--O--O--O-O--O--O--O-O--O--O-O-------------+


                            sched_debug.cpu#11.sched_goidle

  3.5e+06 ++----------------------------------------------------------------+
          |                                .*..  .*                         |
    3e+06 *+.*                      *..  .*    *.  +             *..  .*    |
          |  :                     :   *.           *..*..      :   *.  :   |
  2.5e+06 ++  :     .*             :                      *     :       :  .*
          |   :   *.  :   *.*..*..*                        :   *         *. |
    2e+06 ++  :  +    : ..                                 :  +             |
          |    :+      *                                    :+              |
  1.5e+06 ++   *                                            *               |
          |                                                                 |
    1e+06 ++                                                                |
          |                                                                 |
   500000 ++                                                                |
          |                                                                 |
        0 O+-O-O--O--O-O--O-O--O--O-O--O--O-O--O--O-O--O--O-O--O------------+


                             sched_debug.cpu#11.ttwu_count

  3.5e+06 ++----------------------------------------------------------------+
          |         .*                                 *..                  |
    3e+06 *+.     *. :      *..*..*.     .*.          +          *..*..*    |
          |      :    :    :        *..*.   *..*..   +    *     +       :   *
  2.5e+06 ++ *   :    :    :                      *.*      :   *        : ..|
          |   + :     :   *                                : ..          *  |
    2e+06 ++   *      :  +                                  *               |
          |            :+                                                   |
  1.5e+06 ++           *                                                    |
          |                                                                 |
    1e+06 ++                                                                |
          |                                                                 |
   500000 ++                                                                |
          |                                                                 |
        0 O+-O-O--O--O-O--O-O--O--O-O--O--O-O--O--O-O--O--O-O--O------------+


	[*] bisect-good sample
	[O] bisect-bad  sample

To reproduce:

	apt-get install ruby
	git clone git://git.kernel.org/pub/scm/linux/kernel/git/wfg/lkp-tests.git
	cd lkp-tests
	bin/setup-local job.yaml # the job file attached in this email
	bin/run-local   job.yaml


Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


Thanks,
Huang, Ying


View attachment "job.yaml" of type "text/plain" (1637 bytes)

View attachment "reproduce" of type "text/plain" (41 bytes)

_______________________________________________
LKP mailing list
LKP@...ux.intel.com

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ