lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <20180620073220.GG11011@yexl-desktop>
Date:   Wed, 20 Jun 2018 15:32:20 +0800
From:   kernel test robot <xiaolong.ye@...el.com>
To:     subhra mazumdar <subhra.mazumdar@...cle.com>
Cc:     linux-kernel@...r.kernel.org, peterz@...radead.org,
        mingo@...hat.com, steven.sistare@...cle.com,
        dhaval.giani@...cle.com, rohit.k.jain@...cle.com,
        daniel.lezcano@...aro.org, lkp@...org
Subject: [lkp-robot] [sched]  8dd662615c:  reaim.jobs_per_min -3.3% regression


Greeting,

FYI, we noticed a -3.3% regression of reaim.jobs_per_min due to commit:


commit: 8dd662615c7a4dd4347d89d2f5647cba9c51c2af ("sched: limit cpu search in select_idle_cpu")
url: https://github.com/0day-ci/linux/commits/subhra-mazumdar/Improve-scheduler-scalability-for-fast-path/20180613-015158


in testcase: reaim
on test machine: 72 threads Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz with 128G memory
with following parameters:

	runtime: 300s
	nr_task: 1600%
	test: new_fserver
	cpufreq_governor: performance

test-description: REAIM is an updated and improved version of AIM 7 benchmark.
test-url: https://sourceforge.net/projects/re-aim-7/

In addition to that, the commit also has significant impact on the following tests:

+------------------+-----------------------------------------------------------------------+
| testcase: change | hackbench: hackbench.throughput 12.7% improvement                     |
| test machine     | 72 threads Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz with 128G memory |
| test parameters  | cpufreq_governor=performance                                          |
|                  | ipc=pipe                                                              |
|                  | iterations=12                                                         |
|                  | mode=process                                                          |
|                  | nr_threads=50%                                                        |
+------------------+-----------------------------------------------------------------------+


Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp install job.yaml  # job file is attached in this email
        bin/lkp run     job.yaml

=========================================================================================
compiler/cpufreq_governor/kconfig/nr_task/rootfs/runtime/tbox_group/test/testcase:
  gcc-7/performance/x86_64-rhel-7.2/1600%/debian-x86_64-2018-04-03.cgz/300s/lkp-hsw-ep2/new_fserver/reaim

commit: 
  2539fc82aa ("sched/fair: Update util_est before updating schedutil")
  8dd662615c ("sched: limit cpu search in select_idle_cpu")

2539fc82aa9b07d9 8dd662615c7a4dd4347d89d2f5 
---------------- -------------------------- 
         %stddev     %change         %stddev
             \          |                \  
    938731            -3.3%     908031        reaim.jobs_per_min
    814.87            -3.3%     788.22        reaim.jobs_per_min_child
    948046            -3.3%     916622        reaim.max_jobs_per_min
    141.74            -3.7%     136.50        reaim.child_systime
    297.72            -2.0%     291.73        reaim.child_utime
      7.22            +3.4%       7.46        reaim.parent_time
     14.70            -3.8%      14.14        reaim.std_dev_percent
   5897031            +4.7%    6174085        reaim.time.involuntary_context_switches
 7.527e+08            -3.1%  7.291e+08        reaim.time.minor_page_faults
      4709            -5.0%       4472        reaim.time.percent_of_cpu_this_job_got
      4683            -6.6%       4373        reaim.time.system_time
      9825            -5.0%       9336        reaim.time.user_time
  13502630 ±  2%     -12.2%   11859024        reaim.time.voluntary_context_switches
   3801600            -3.0%    3686400        reaim.workload
      0.00 ±  9%      +0.0        0.00 ± 16%  mpstat.cpu.iowait%
    125038            -8.9%     113866        vmstat.system.cs
     76044            +2.6%      78021        vmstat.system.in
     86274 ±  6%     -16.7%      71862 ±  9%  numa-vmstat.node1.nr_active_anon
     32499 ± 22%     -28.7%      23186 ± 30%  numa-vmstat.node1.nr_slab_unreclaimable
     86274 ±  6%     -16.7%      71862 ±  9%  numa-vmstat.node1.nr_zone_active_anon
    344856 ±  6%     -16.6%     287582 ±  9%  numa-meminfo.node1.Active
    344854 ±  6%     -16.6%     287580 ±  9%  numa-meminfo.node1.Active(anon)
    129963 ± 22%     -28.7%      92645 ± 30%  numa-meminfo.node1.SUnreclaim
    157613 ± 17%     -22.8%     121642 ± 24%  numa-meminfo.node1.Slab
    106884           -19.6%      85936        slabinfo.kmalloc-32.active_objs
    836.25           -19.8%     670.75        slabinfo.kmalloc-32.active_slabs
    107116           -19.8%      85955        slabinfo.kmalloc-32.num_objs
    836.25           -19.8%     670.75        slabinfo.kmalloc-32.num_slabs
 7.012e+08            -3.1%  6.792e+08        proc-vmstat.numa_hit
 7.012e+08            -3.1%  6.792e+08        proc-vmstat.numa_local
     14167            +3.0%      14590        proc-vmstat.numa_other
 7.123e+08            -3.2%  6.894e+08        proc-vmstat.pgalloc_normal
 7.539e+08            -3.1%  7.303e+08        proc-vmstat.pgfault
 7.123e+08            -3.2%  6.894e+08        proc-vmstat.pgfree
    262.50 ± 13%     +18.6%     311.29 ±  6%  sched_debug.cfs_rq:/.removed.util_avg.max
    497313 ± 12%     +26.5%     629284 ±  4%  sched_debug.cpu.avg_idle.avg
     33703 ± 55%    +400.3%     168620 ± 51%  sched_debug.cpu.avg_idle.min
     38227 ±  5%     +18.7%      45393 ±  2%  sched_debug.cpu.curr->pid.max
     10018 ± 18%     +41.9%      14220 ± 16%  sched_debug.cpu.curr->pid.stddev
    309090 ± 26%     -51.9%     148822 ± 45%  sched_debug.cpu.load.max
      2427 ± 18%     -31.8%       1656 ± 12%  sched_debug.cpu.sched_goidle.stddev
     45654 ±  3%     +22.4%      55880 ± 16%  sched_debug.cpu.ttwu_local.max
 2.396e+08 ±  5%     -35.0%  1.557e+08 ±  5%  cpuidle.C1.time
   3257365 ±  7%     -42.8%    1863490 ±  8%  cpuidle.C1.usage
 2.834e+08           -29.8%   1.99e+08        cpuidle.C1E.time
   2727675           -31.3%    1874186        cpuidle.C1E.usage
   5580441            -9.4%    5057704        cpuidle.C3.usage
 5.676e+09           +12.9%   6.41e+09        cpuidle.C6.time
   6613043           +21.6%    8042531        cpuidle.C6.usage
   1459318 ±  3%     -10.8%    1301821 ±  2%  cpuidle.POLL.time
     61461 ±  7%     -29.9%      43098 ±  6%  cpuidle.POLL.usage
      1853            -4.8%       1765        turbostat.Avg_MHz
   3252668 ±  7%     -42.9%    1857319 ±  8%  turbostat.C1
      1.07 ±  4%      -0.4        0.70 ±  5%  turbostat.C1%
   2725753           -31.3%    1872752        turbostat.C1E
      1.27            -0.4        0.90        turbostat.C1E%
   5579952            -9.4%    5057162        turbostat.C3
   6610452           +21.6%    8040209        turbostat.C6
     25.48            +3.4       28.92        turbostat.C6%
     15.88           +21.7%      19.33        turbostat.CPU%c1
      0.26 ±  3%     +45.7%       0.38 ±  2%  turbostat.CPU%c3
 5.215e+12            -3.4%  5.039e+12        perf-stat.branch-instructions
      1.23            +0.0        1.25        perf-stat.branch-miss-rate%
 6.392e+10            -1.4%  6.304e+10        perf-stat.branch-misses
      6.68            +0.1        6.81        perf-stat.cache-miss-rate%
 1.894e+11            -3.3%  1.831e+11        perf-stat.cache-references
  38848096            -9.5%   35176187        perf-stat.context-switches
      1.23            -2.1%       1.20        perf-stat.cpi
   4.1e+13            -5.3%  3.884e+13        perf-stat.cpu-cycles
  11806401           -10.8%   10528657        perf-stat.cpu-migrations
 1.092e+10 ±  4%     -11.2%  9.699e+09 ±  5%  perf-stat.dTLB-load-misses
 6.639e+12            -3.3%  6.417e+12        perf-stat.dTLB-loads
 2.865e+12            -3.2%  2.773e+12        perf-stat.dTLB-stores
     54.31            -0.4       53.86        perf-stat.iTLB-load-miss-rate%
 6.194e+09            -5.2%  5.875e+09        perf-stat.iTLB-load-misses
 5.212e+09            -3.4%  5.033e+09        perf-stat.iTLB-loads
 3.331e+13            -3.2%  3.224e+13        perf-stat.instructions
      0.81            +2.2%       0.83        perf-stat.ipc
 7.453e+08            -3.1%   7.22e+08        perf-stat.minor-faults
     87.48            +2.1       89.56        perf-stat.node-load-miss-rate%
 1.141e+09           -17.5%  9.416e+08 ±  2%  perf-stat.node-loads
     55.05            +1.3       56.38        perf-stat.node-store-miss-rate%
 1.561e+09            -4.9%  1.485e+09        perf-stat.node-stores
 7.453e+08            -3.1%   7.22e+08        perf-stat.page-faults
      1.88 ± 13%      -0.8        1.10 ±  7%  perf-profile.calltrace.cycles-pp.down_write.path_openat.do_filp_open.do_sys_open.do_syscall_64
      1.73 ± 14%      -0.7        1.00 ±  7%  perf-profile.calltrace.cycles-pp.call_rwsem_down_write_failed.down_write.path_openat.do_filp_open.do_sys_open
      1.73 ± 14%      -0.7        1.00 ±  7%  perf-profile.calltrace.cycles-pp.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write.path_openat.do_filp_open
      7.29 ±  2%      -0.7        6.61 ±  3%  perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.22 ± 17%      -0.6        0.63 ±  6%  perf-profile.calltrace.cycles-pp.osq_lock.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write.path_openat
      0.70 ±  6%      -0.3        0.41 ± 58%  perf-profile.calltrace.cycles-pp.__split_vma.do_munmap.__x64_sys_brk.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.10 ±  5%      -0.3        0.82 ±  3%  perf-profile.calltrace.cycles-pp.elf_map.load_elf_binary.search_binary_handler.do_execveat_common.__x64_sys_execve
      1.91 ±  4%      -0.3        1.63 ±  4%  perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.release_pages.tlb_flush_mmu_free.arch_tlb_finish_mmu.tlb_finish_mmu
      1.41 ±  5%      -0.3        1.14 ±  3%  perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary
      1.40 ±  5%      -0.3        1.14 ±  3%  perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry
      0.94 ±  3%      -0.3        0.69 ±  4%  perf-profile.calltrace.cycles-pp.up_write.path_openat.do_filp_open.do_sys_open.do_syscall_64
      1.75 ±  4%      -0.3        1.50 ±  4%  perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.release_pages.tlb_flush_mmu_free.arch_tlb_finish_mmu
      0.87            -0.2        0.64 ±  5%  perf-profile.calltrace.cycles-pp.rwsem_wake.call_rwsem_wake.up_write.path_openat.do_filp_open
      0.87            -0.2        0.64 ±  5%  perf-profile.calltrace.cycles-pp.call_rwsem_wake.up_write.path_openat.do_filp_open.do_sys_open
      2.12 ±  3%      -0.1        2.01 ±  2%  perf-profile.calltrace.cycles-pp.setlocale
      0.75 ±  6%      -0.1        0.67 ±  4%  perf-profile.calltrace.cycles-pp.iterate_dir.__x64_sys_getdents.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.70 ±  6%      -0.1        0.62 ±  3%  perf-profile.calltrace.cycles-pp.dcache_readdir.iterate_dir.__x64_sys_getdents.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.76 ±  6%      -0.1        0.68 ±  4%  perf-profile.calltrace.cycles-pp.__x64_sys_getdents.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.89 ±  3%      -0.1        0.83 ±  3%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.setlocale
      0.88 ±  2%      -0.1        0.82 ±  3%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.setlocale
      0.69            -0.0        0.66 ±  3%  perf-profile.calltrace.cycles-pp.smpboot_thread_fn.kthread.ret_from_fork
      0.77 ±  4%      +0.1        0.85 ±  4%  perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt
      0.79 ±  5%      +0.1        0.87 ±  4%  perf-profile.calltrace.cycles-pp.apic_timer_interrupt
     23.64            +2.1       25.74        perf-profile.calltrace.cycles-pp.secondary_startup_64
     23.36            +2.4       25.73        perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
     23.36            +2.4       25.73        perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64
     23.36            +2.4       25.73        perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64
     22.43            +2.6       25.01        perf-profile.calltrace.cycles-pp.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
     20.81            +3.0       23.78        perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary
     46.88 ±  2%      -1.8       45.06        perf-profile.children.cycles-pp.do_syscall_64
     47.07 ±  2%      -1.8       45.25        perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
      7.94 ±  3%      -0.8        7.11 ±  2%  perf-profile.children.cycles-pp.path_openat
      7.95 ±  3%      -0.8        7.14 ±  2%  perf-profile.children.cycles-pp.do_filp_open
      8.14 ±  3%      -0.8        7.37 ±  3%  perf-profile.children.cycles-pp.do_sys_open
      4.23 ±  3%      -0.7        3.54 ±  5%  perf-profile.children.cycles-pp.down_write
      3.33 ±  6%      -0.7        2.67 ±  7%  perf-profile.children.cycles-pp.rwsem_down_write_failed
      3.34 ±  6%      -0.7        2.69 ±  7%  perf-profile.children.cycles-pp.call_rwsem_down_write_failed
      2.08 ± 10%      -0.6        1.50 ±  9%  perf-profile.children.cycles-pp.osq_lock
      2.86 ±  2%      -0.2        2.69 ±  2%  perf-profile.children.cycles-pp.apic_timer_interrupt
      2.84 ±  2%      -0.2        2.67 ±  2%  perf-profile.children.cycles-pp.smp_apic_timer_interrupt
      0.91 ±  3%      -0.2        0.74 ±  5%  perf-profile.children.cycles-pp.do_unlinkat
      1.28 ±  3%      -0.2        1.12 ±  5%  perf-profile.children.cycles-pp.hrtimer_interrupt
      1.06 ±  3%      -0.1        0.94 ±  5%  perf-profile.children.cycles-pp.__hrtimer_run_queues
      0.41 ±  7%      -0.1        0.29 ± 10%  perf-profile.children.cycles-pp.menu_select
      0.60 ±  5%      -0.1        0.48 ±  4%  perf-profile.children.cycles-pp.do_linkat
      0.60 ±  5%      -0.1        0.49 ±  4%  perf-profile.children.cycles-pp.__x64_sys_link
      2.12 ±  3%      -0.1        2.01 ±  2%  perf-profile.children.cycles-pp.setlocale
      1.17            -0.1        1.07 ±  2%  perf-profile.children.cycles-pp.call_rwsem_wake
      1.18            -0.1        1.07 ±  2%  perf-profile.children.cycles-pp.rwsem_wake
      0.34 ±  8%      -0.1        0.23 ±  4%  perf-profile.children.cycles-pp.filename_create
      1.75            -0.1        1.65        perf-profile.children.cycles-pp.up_write
      0.73            -0.1        0.65 ±  7%  perf-profile.children.cycles-pp.tick_sched_timer
      0.89 ±  3%      -0.1        0.80 ±  2%  perf-profile.children.cycles-pp.wake_up_q
      0.72 ±  7%      -0.1        0.64 ±  4%  perf-profile.children.cycles-pp.dcache_readdir
      1.14            -0.1        1.05        perf-profile.children.cycles-pp.exit_to_usermode_loop
      0.75 ±  6%      -0.1        0.67 ±  4%  perf-profile.children.cycles-pp.iterate_dir
      0.76 ±  6%      -0.1        0.68 ±  4%  perf-profile.children.cycles-pp.__x64_sys_getdents
      0.98            -0.1        0.91 ±  2%  perf-profile.children.cycles-pp.copy_strings
      0.19 ± 16%      -0.1        0.12 ± 11%  perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
      0.63 ±  4%      -0.1        0.56 ±  3%  perf-profile.children.cycles-pp.__fput
      0.74 ±  4%      -0.1        0.67        perf-profile.children.cycles-pp.task_work_run
      0.31 ±  5%      -0.1        0.24 ±  5%  perf-profile.children.cycles-pp.run_ksoftirqd
      0.44 ±  4%      -0.1        0.37 ±  5%  perf-profile.children.cycles-pp.__might_fault
      0.27 ± 19%      -0.1        0.21 ± 11%  perf-profile.children.cycles-pp.memcpy_erms
      0.16 ± 22%      -0.1        0.10 ± 27%  perf-profile.children.cycles-pp.ktime_get
      0.16 ± 14%      -0.1        0.10 ± 10%  perf-profile.children.cycles-pp.tick_nohz_next_event
      0.66            -0.1        0.61 ±  6%  perf-profile.children.cycles-pp.tick_sched_handle
      1.04 ±  4%      -0.1        0.99        perf-profile.children.cycles-pp.kmem_cache_free
      0.41 ±  7%      -0.0        0.36 ±  6%  perf-profile.children.cycles-pp._copy_to_user
      0.43 ±  5%      -0.0        0.39 ±  4%  perf-profile.children.cycles-pp.page_add_file_rmap
      0.21 ±  7%      -0.0        0.17 ±  2%  perf-profile.children.cycles-pp.schedule_idle
      0.51            -0.0        0.47        perf-profile.children.cycles-pp.do_signal
      0.65            -0.0        0.61 ±  3%  perf-profile.children.cycles-pp.get_empty_filp
      0.14 ±  7%      -0.0        0.11 ±  6%  perf-profile.children.cycles-pp.selinux_vm_enough_memory
      0.21 ±  2%      -0.0        0.17 ±  6%  perf-profile.children.cycles-pp.rebalance_domains
      0.10 ± 11%      -0.0        0.07 ± 17%  perf-profile.children.cycles-pp.get_next_timer_interrupt
      0.36 ±  4%      -0.0        0.33 ±  2%  perf-profile.children.cycles-pp.do_open_execat
      0.14 ±  6%      -0.0        0.11 ±  6%  perf-profile.children.cycles-pp.__slab_alloc
      0.21 ±  8%      -0.0        0.18 ±  3%  perf-profile.children.cycles-pp.__pmd_alloc
      0.16 ±  6%      -0.0        0.14 ±  3%  perf-profile.children.cycles-pp.filp_close
      0.17 ±  6%      -0.0        0.15 ± 10%  perf-profile.children.cycles-pp.get_user_arg_ptr
      0.16 ±  5%      -0.0        0.13 ±  6%  perf-profile.children.cycles-pp.free_pcppages_bulk
      0.12 ± 10%      -0.0        0.10 ±  7%  perf-profile.children.cycles-pp.___slab_alloc
      0.24 ±  5%      -0.0        0.22        perf-profile.children.cycles-pp.copy_strings_kernel
      0.10 ±  8%      -0.0        0.08 ±  5%  perf-profile.children.cycles-pp.irq_enter
      0.09 ±  4%      -0.0        0.08 ±  5%  perf-profile.children.cycles-pp.run_rebalance_domains
      0.06            -0.0        0.05        perf-profile.children.cycles-pp.__d_lookup_done
      0.10 ±  4%      +0.0        0.12 ±  5%  perf-profile.children.cycles-pp.__update_load_avg_se
      0.07 ± 12%      +0.0        0.09        perf-profile.children.cycles-pp.__task_pid_nr_ns
      0.34 ±  2%      +0.0        0.36 ±  2%  perf-profile.children.cycles-pp.unlock_page
      0.04 ± 58%      +0.0        0.07 ± 10%  perf-profile.children.cycles-pp.rwsem_down_read_failed
      0.04 ± 58%      +0.0        0.07 ± 10%  perf-profile.children.cycles-pp.call_rwsem_down_read_failed
      0.04 ± 58%      +0.0        0.08 ± 10%  perf-profile.children.cycles-pp.cpumask_next_and
      0.00            +0.1        0.05 ±  8%  perf-profile.children.cycles-pp.reschedule_interrupt
      0.00            +0.1        0.06 ± 11%  perf-profile.children.cycles-pp.find_next_and_bit
      1.73            +0.1        1.80        perf-profile.children.cycles-pp.flush_tlb_func_common
      1.14            +0.1        1.21 ±  2%  perf-profile.children.cycles-pp.rcu_process_callbacks
      0.43 ±  6%      +0.1        0.51 ±  4%  perf-profile.children.cycles-pp.wait4
      0.22 ±  8%      +0.1        0.30 ±  3%  perf-profile.children.cycles-pp.do_task_dead
      0.62 ±  5%      +0.1        0.71 ±  4%  perf-profile.children.cycles-pp.kernel_wait4
      0.61 ±  5%      +0.1        0.69 ±  4%  perf-profile.children.cycles-pp.do_wait
      0.62 ±  5%      +0.1        0.71 ±  4%  perf-profile.children.cycles-pp.__do_sys_wait4
      0.26 ±  3%      +0.1        0.36 ±  4%  perf-profile.children.cycles-pp._atomic_dec_and_lock
      0.28 ±  5%      +0.1        0.39 ±  3%  perf-profile.children.cycles-pp.free_uid
      0.34 ±  4%      +0.1        0.47 ±  2%  perf-profile.children.cycles-pp.put_cred_rcu
      0.34 ±  2%      +0.2        0.51 ±  3%  perf-profile.children.cycles-pp.find_busiest_group
      0.77 ±  2%      +0.2        0.95        perf-profile.children.cycles-pp.schedule
      1.32            +0.2        1.54        perf-profile.children.cycles-pp.__schedule
      0.61 ±  4%      +0.2        0.83 ±  2%  perf-profile.children.cycles-pp.load_balance
      0.62 ±  3%      +0.3        0.89        perf-profile.children.cycles-pp.pick_next_task_fair
     23.65            +2.1       25.75        perf-profile.children.cycles-pp.do_idle
     23.64            +2.1       25.74        perf-profile.children.cycles-pp.cpu_startup_entry
     23.64            +2.1       25.74        perf-profile.children.cycles-pp.secondary_startup_64
     22.75            +2.3       25.07        perf-profile.children.cycles-pp.cpuidle_enter_state
     23.36            +2.4       25.73        perf-profile.children.cycles-pp.start_secondary
     21.06            +2.7       23.79        perf-profile.children.cycles-pp.intel_idle
      2.07 ± 10%      -0.6        1.48 ±  9%  perf-profile.self.cycles-pp.osq_lock
      0.27 ± 18%      -0.1        0.21 ±  9%  perf-profile.self.cycles-pp.memcpy_erms
      0.11 ± 40%      -0.1        0.05 ± 74%  perf-profile.self.cycles-pp.ktime_get
      0.16 ±  5%      -0.0        0.12 ± 19%  perf-profile.self.cycles-pp.menu_select
      0.36 ±  4%      -0.0        0.32 ±  4%  perf-profile.self.cycles-pp.page_add_file_rmap
      0.48            -0.0        0.46 ±  2%  perf-profile.self.cycles-pp.__slab_free
      0.12 ±  7%      -0.0        0.10 ±  5%  perf-profile.self.cycles-pp.cpuidle_enter_state
      0.09 ±  7%      -0.0        0.07 ± 12%  perf-profile.self.cycles-pp.free_pcppages_bulk
      0.07 ± 11%      -0.0        0.05 ±  8%  perf-profile.self.cycles-pp.do_idle
      0.23 ±  4%      -0.0        0.21 ±  4%  perf-profile.self.cycles-pp.update_load_avg
      0.15 ±  6%      -0.0        0.13 ±  3%  perf-profile.self.cycles-pp.do_dentry_open
      0.08 ± 12%      -0.0        0.06 ±  6%  perf-profile.self.cycles-pp.update_rq_clock
      0.08 ±  6%      -0.0        0.06 ±  6%  perf-profile.self.cycles-pp.security_inode_permission
      0.07 ±  6%      +0.0        0.08 ±  5%  perf-profile.self.cycles-pp.load_balance
      0.34 ±  2%      +0.0        0.36 ±  3%  perf-profile.self.cycles-pp.unlock_page
      0.07 ± 12%      +0.0        0.09        perf-profile.self.cycles-pp.__task_pid_nr_ns
      0.15 ±  3%      +0.0        0.17 ±  4%  perf-profile.self.cycles-pp.anon_vma_clone
      0.08 ± 13%      +0.0        0.12 ± 13%  perf-profile.self.cycles-pp.try_to_wake_up
      0.01 ±173%      +0.0        0.06 ± 11%  perf-profile.self.cycles-pp.pick_next_task_fair
      0.00            +0.1        0.06 ± 11%  perf-profile.self.cycles-pp.find_next_and_bit
      0.26 ±  3%      +0.1        0.36 ±  4%  perf-profile.self.cycles-pp._atomic_dec_and_lock
      0.28 ±  2%      +0.1        0.42 ±  4%  perf-profile.self.cycles-pp.find_busiest_group
     21.05            +2.7       23.79        perf-profile.self.cycles-pp.intel_idle


                                                                                
                                 reaim.parent_time                              
                                                                                
   7.5 +-+------------------------------------------------------O-----------+   
       O    O O  O   O              O                         O             |   
  7.45 +-O         O    O O    O O           O         O   O                |   
       |                    O         O O  O   O  O O    O                  |   
   7.4 +-+                                                                  |   
       |                                                                    |   
  7.35 +-+                                                                  |   
       |                                                                    |   
   7.3 +-+                                                                  |   
       |           +.  .+       .+..           +..                          |   
  7.25 +-+  +.    +  +.  + .+..+    +.        +                        +    |   
       |:  +  +..+        +           +.+..+.+    +.+..+.+.     +.    + +   |   
   7.2 +-++                                                +.. +  +..+   + +|   
       | +                                                    +           + |   
  7.15 +-+------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                 reaim.child_utime                              
                                                                                
  300 +-+-------------------------------------------------------------------+   
      |                                .+.  .+                              |   
  299 +-+          .+..      .+.. .+.+.   +.  +                             |   
  298 +-+..      .+    +.+..+    +             +.  .+.  .+.+..+. .+..+.  .+.|   
      |    +.+..+                                +.   +.        +      +.   |   
  297 +-+                                                                   |   
  296 +-+                                                                   |   
      |                                                                     |   
  295 +-+                                                                   |   
  294 +-+                   O                                               |   
      | O    O    O      O    O    O                                        |   
  293 O-+  O    O   O  O         O   O  O O  O O O  O O                     |   
  292 +-+                                                O                  |   
      |                                                    O  O O           |   
  291 +-+-------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                 reaim.jobs_per_min                             
                                                                                
  945000 +-+----------------------------------------------------------------+   
         |: :                                              .+.+     .+    + |   
  940000 +-+ :             +..        .+.+..+.+    +.+.+..+    +  .+  + .. +|   
  935000 +-+ : .+.+..     +    .+.  .+         :  +             +.     +    |   
         |    +      +.+.+    +   +.           : +                          |   
  930000 +-+                                    +                           |   
  925000 +-+                                                                |   
         |                                                                  |   
  920000 +-+                                                                |   
  915000 +-+                                                                |   
         |                             O O  O   O  O O    O                 |   
  910000 +-O             O O  O O O           O        O    O               |   
  905000 O-+  O O O  O O             O                        O             |   
         |                                                      O           |   
  900000 +-+----------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                             reaim.jobs_per_min_child                           
                                                                                
  820 +-+-------------------------------------------------------------------+   
      | ::                                                .+.  :          + |   
  815 +-+ :              +..           .+.  .+   +..+.+..+     : .+..+. .. +|   
      |:  : .+..+       :           .+.   +.  : :               +      +    |   
  810 +-+  +     + .+.. :   +.+..+.+          : :                           |   
      |           +    +                       +                            |   
  805 +-+                                                                   |   
      |                                                                     |   
  800 +-+                                                                   |   
      |                                                                     |   
  795 +-+                                                                   |   
      |                              O  O O    O O  O    O                  |   
  790 +-O              O O  O    O           O        O    O                |   
      O      O    O O         O    O                          O             |   
  785 +-+--O----O-----------------------------------------------O-----------+   
                                                                                
                                                                                                                                                                
                      reaim.time.percent_of_cpu_this_job_got                    
                                                                                
  4750 +-+------------------------------------------------------------------+   
       |.+..             .+.+.. .+..+.+.+..+.+.              .+. .+..    .+.|   
  4700 +-+   .+..+.+.+..+      +               +..+.+..+.+.+.   +    +.+.   |   
       |    +                                                               |   
       |                                                                    |   
  4650 +-+                                                                  |   
       |                                                                    |   
  4600 +-+                                                                  |   
       |                                                                    |   
  4550 +-+                                                                  |   
       O O  O O  O O O  O O O  O O  O                                       |   
       |                                O  O      O O                       |   
  4500 +-+                            O      O O       O                    |   
       |                                                 O O  O O           |   
  4450 +-+------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                        reaim.time.involuntary_context_switches                 
                                                                                
   6.2e+06 +-+--------------------------------------------O-----------------+   
  6.15e+06 +-+                                              O  O O          |   
           |                        O                                       |   
   6.1e+06 O-O O  O O O O  O O O O    O                                     |   
  6.05e+06 +-+                          O O  O O O O  O O                   |   
           |                                                                |   
     6e+06 +-+                                                              |   
  5.95e+06 +-+                                                              |   
   5.9e+06 +-+.    .+.             .+. .+.+.. .+              .+. .+.+..+.+.|   
           |   +..+   +    +.+.+.+.   +      + :   +..+.+.+.+.   +          |   
  5.85e+06 +-+        :   :                     :  :                        |   
   5.8e+06 +-+         :  :                     : :                         |   
           |           : :                       ::                         |   
  5.75e+06 +-+          ::                       +                          |   
   5.7e+06 +-+--------------------------------------------------------------+   
                                                                                
                                                                                
[*] bisect-good sample
[O] bisect-bad  sample

***************************************************************************************************
lkp-hsw-ep4: 72 threads Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz with 128G memory
=========================================================================================
compiler/cpufreq_governor/ipc/iterations/kconfig/mode/nr_threads/rootfs/tbox_group/testcase:
  gcc-7/performance/pipe/12/x86_64-rhel-7.2/process/50%/debian-x86_64-2018-04-03.cgz/lkp-hsw-ep4/hackbench

commit: 
  2539fc82aa ("sched/fair: Update util_est before updating schedutil")
  8dd662615c ("sched: limit cpu search in select_idle_cpu")

2539fc82aa9b07d9 8dd662615c7a4dd4347d89d2f5 
---------------- -------------------------- 
         %stddev     %change         %stddev
             \          |                \  
    123935           +12.7%     139696        hackbench.throughput
    818.39           -11.3%     726.18        hackbench.time.elapsed_time
    818.39           -11.3%     726.18        hackbench.time.elapsed_time.max
  4.18e+09 ±  2%     -21.1%  3.297e+09        hackbench.time.involuntary_context_switches
     52401           -11.6%      46313        hackbench.time.system_time
      6201            -9.6%       5606        hackbench.time.user_time
 6.222e+09           -12.1%  5.467e+09        hackbench.time.voluntary_context_switches
      0.64 ±  3%      +0.2        0.81 ±  7%  mpstat.cpu.idle%
      2871 ±  6%      +8.7%       3120 ±  4%  slabinfo.sock_inode_cache.active_objs
      2871 ±  6%      +8.7%       3120 ±  4%  slabinfo.sock_inode_cache.num_objs
   9642032 ±  4%     -14.5%    8247745 ±  2%  softirqs.RCU
    831999 ±  2%     +13.0%     939983        softirqs.SCHED
  12709169            -5.0%   12067441        vmstat.system.cs
    973111           +16.2%    1130820        vmstat.system.in
  66696250 ±  5%     +49.2%   99499793 ±  4%  cpuidle.C1.time
  11551951 ±  6%     +96.9%   22743914 ±  3%  cpuidle.C1.usage
    844303 ±  6%    +139.2%    2019174 ±  2%  cpuidle.POLL.time
    182674 ±  8%    +129.8%     419855 ±  3%  cpuidle.POLL.usage
  27699444 ±  5%     +28.2%   35505978 ±  2%  numa-numastat.node0.local_node
  27702926 ±  5%     +28.2%   35511779 ±  2%  numa-numastat.node0.numa_hit
  30815427           +18.8%   36597997        numa-numastat.node1.local_node
  30826044           +18.8%   36606343        numa-numastat.node1.numa_hit
  14003985 ±  6%     +30.7%   18298916 ±  3%  numa-vmstat.node0.numa_hit
  14000286 ±  6%     +30.7%   18292925 ±  3%  numa-vmstat.node0.numa_local
  15401529           +20.7%   18585831 ±  2%  numa-vmstat.node1.numa_hit
  15255090           +20.9%   18441285 ±  2%  numa-vmstat.node1.numa_local
  11550825 ±  6%     +96.9%   22743029 ±  3%  turbostat.C1
      0.11 ±  4%      +0.1        0.19 ±  4%  turbostat.C1%
      0.35 ±  3%      +0.1        0.44 ± 14%  turbostat.C6%
      0.47 ±  2%     +20.9%       0.57 ±  4%  turbostat.CPU%c1
      0.23 ±  3%     +27.7%       0.30 ± 15%  turbostat.CPU%c6
     47567            +1.0%      48036        proc-vmstat.nr_slab_unreclaimable
  58537822 ±  2%     +23.2%   72134162        proc-vmstat.numa_hit
  58523716 ±  2%     +23.2%   72120007        proc-vmstat.numa_local
  58726400 ±  2%     +23.1%   72307125        proc-vmstat.pgalloc_normal
   2444733            -7.2%    2268341        proc-vmstat.pgfault
  58669621 ±  2%     +23.2%   72252852        proc-vmstat.pgfree
  1.45e+13           -10.9%  1.292e+13        perf-stat.branch-instructions
      1.78            +0.1        1.85        perf-stat.branch-miss-rate%
 2.579e+11            -7.2%  2.393e+11        perf-stat.branch-misses
      0.87 ±  5%      -0.1        0.79 ±  4%  perf-stat.cache-miss-rate%
 8.286e+09 ±  7%     -23.0%  6.378e+09 ±  4%  perf-stat.cache-misses
 9.472e+11           -14.7%  8.078e+11        perf-stat.cache-references
 1.043e+10           -15.6%  8.797e+09        perf-stat.context-switches
 1.424e+14           -11.2%  1.264e+14        perf-stat.cpu-cycles
 2.421e+09 ±  3%     -31.3%  1.663e+09        perf-stat.cpu-migrations
      1.71 ±  3%      -0.3        1.43 ± 12%  perf-stat.dTLB-load-miss-rate%
 3.701e+11 ±  2%     -25.0%  2.774e+11 ± 12%  perf-stat.dTLB-load-misses
 2.126e+13            -9.7%  1.919e+13        perf-stat.dTLB-loads
      0.11 ±  8%      -0.0        0.09 ± 13%  perf-stat.dTLB-store-miss-rate%
 1.503e+10 ±  7%     -28.0%  1.082e+10 ± 13%  perf-stat.dTLB-store-misses
 1.313e+13            -9.5%  1.187e+13        perf-stat.dTLB-stores
     55.10            +1.2       56.30        perf-stat.iTLB-load-miss-rate%
 1.164e+11 ±  2%     -10.3%  1.044e+11        perf-stat.iTLB-load-misses
  9.48e+10           -14.5%  8.103e+10        perf-stat.iTLB-loads
 7.393e+13           -10.7%  6.605e+13        perf-stat.instructions
   2408870            -7.2%    2235758        perf-stat.minor-faults
 5.089e+09 ±  8%     -25.3%  3.801e+09 ±  4%  perf-stat.node-load-misses
  1.95e+09 ±  7%     -18.7%  1.585e+09 ±  3%  perf-stat.node-store-misses
 1.193e+09 ±  6%     -22.2%  9.283e+08 ±  7%  perf-stat.node-stores
   2408870            -7.2%    2235771        perf-stat.page-faults
     71308           -10.7%      63709        perf-stat.path-length
      0.00        +6.6e+12%      66155 ± 93%  sched_debug.cfs_rq:/.MIN_vruntime.avg
      0.00          +3e+14%    2951855 ± 54%  sched_debug.cfs_rq:/.MIN_vruntime.max
      0.00        +2.8e+28%     418846 ± 70%  sched_debug.cfs_rq:/.MIN_vruntime.stddev
     30945 ±  5%     +71.6%      53105 ± 65%  sched_debug.cfs_rq:/.load.max
      5000 ±  2%     +55.4%       7768 ± 51%  sched_debug.cfs_rq:/.load.stddev
      0.00        +6.6e+12%      66155 ± 93%  sched_debug.cfs_rq:/.max_vruntime.avg
      0.00          +3e+14%    2951855 ± 54%  sched_debug.cfs_rq:/.max_vruntime.max
      0.00        +2.8e+28%     418846 ± 70%  sched_debug.cfs_rq:/.max_vruntime.stddev
  29631852           -11.3%   26273202 ±  3%  sched_debug.cfs_rq:/.min_vruntime.max
  26964419           -11.4%   23885494 ±  3%  sched_debug.cfs_rq:/.min_vruntime.min
     31.32 ±  4%     +10.9%      34.73 ±  2%  sched_debug.cfs_rq:/.runnable_load_avg.max
      2.30 ± 19%     -45.1%       1.26 ± 33%  sched_debug.cfs_rq:/.runnable_load_avg.min
      5.38 ±  4%     +14.2%       6.14 ±  3%  sched_debug.cfs_rq:/.runnable_load_avg.stddev
     29759 ±  2%     +76.9%      52656 ± 65%  sched_debug.cfs_rq:/.runnable_weight.max
      5353 ±  3%     +53.7%       8227 ± 48%  sched_debug.cfs_rq:/.runnable_weight.stddev
    127.80 ±  6%     -27.7%      92.36 ± 14%  sched_debug.cfs_rq:/.util_est_enqueued.min
      3.68 ±  9%     -24.7%       2.77 ±  9%  sched_debug.cpu.cpu_load[0].min
      5.29 ±  4%     +13.0%       5.98 ±  4%  sched_debug.cpu.cpu_load[0].stddev
     29.43 ±  4%     +16.5%      34.27 ±  7%  sched_debug.cpu.cpu_load[1].max
      4.55 ±  4%     -23.3%       3.49 ±  9%  sched_debug.cpu.cpu_load[1].min
      4.74 ±  3%     +18.4%       5.62 ±  4%  sched_debug.cpu.cpu_load[1].stddev
      4.88 ±  4%     -18.7%       3.96 ±  8%  sched_debug.cpu.cpu_load[2].min
      4.96 ±  5%     -11.8%       4.38 ±  6%  sched_debug.cpu.cpu_load[3].min
      4.38 ±  4%     +13.8%       4.99 ±  7%  sched_debug.cpu.cpu_load[3].stddev
      2.57 ±  7%     -20.6%       2.04 ±  6%  sched_debug.cpu.nr_running.min
      3.38 ±  3%      +7.4%       3.63 ±  4%  sched_debug.cpu.nr_running.stddev
  70365771           -14.7%   60002575 ±  4%  sched_debug.cpu.nr_switches.avg
  74366842           -15.4%   62919235 ±  4%  sched_debug.cpu.nr_switches.max
  61887561           -14.7%   52785308 ±  3%  sched_debug.cpu.nr_switches.min
      6.11 ± 12%      -2.0        4.11 ±  2%  perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
     22.94 ±  3%      -1.9       21.02        perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.__vfs_write
     23.59 ±  3%      -1.9       21.68        perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.__vfs_write.vfs_write
     22.70 ±  3%      -1.9       20.79        perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write
      4.24 ± 17%      -1.9        2.36        perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common
     25.82 ±  3%      -1.8       24.06        perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.__vfs_write.vfs_write.ksys_write
      2.50 ± 23%      -1.6        0.91        perf-profile.calltrace.cycles-pp.available_idle_cpu.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function
     79.22            -1.1       78.10        perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
     80.03            -1.0       79.01        perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
      6.77 ±  2%      -1.0        5.80 ±  2%  perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
      6.80 ±  2%      -0.9        5.86 ±  2%  perf-profile.calltrace.cycles-pp.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
      6.95 ±  2%      -0.9        6.02 ±  2%  perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
      2.44            -0.4        2.00 ±  3%  perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__schedule.schedule.exit_to_usermode_loop.do_syscall_64
      1.31 ±  6%      -0.2        1.07 ±  6%  perf-profile.calltrace.cycles-pp.set_task_cpu.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
      2.48            -0.2        2.26        perf-profile.calltrace.cycles-pp.__switch_to
      1.61 ±  4%      -0.2        1.42 ±  5%  perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_read.ksys_read.do_syscall_64
      1.82 ±  2%      -0.2        1.65        perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop.do_syscall_64
      1.51            -0.1        1.37 ±  2%  perf-profile.calltrace.cycles-pp.native_write_msr
      1.06            -0.1        0.94 ±  2%  perf-profile.calltrace.cycles-pp.load_new_mm_cr3.switch_mm_irqs_off.__schedule.schedule.exit_to_usermode_loop
      1.02 ±  6%      -0.1        0.90 ±  5%  perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__schedule.schedule
      0.96 ±  5%      -0.1        0.84 ±  5%  perf-profile.calltrace.cycles-pp.___perf_sw_event.__schedule.schedule.pipe_wait.pipe_read
      0.72 ±  3%      -0.1        0.65 ±  2%  perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.dequeue_task_fair.__schedule.schedule
      0.70 ±  3%      -0.1        0.62        perf-profile.calltrace.cycles-pp.put_prev_entity.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop
      0.83 ±  4%      -0.1        0.76        perf-profile.calltrace.cycles-pp.finish_task_switch.__schedule.schedule.pipe_wait.pipe_read
      0.76            -0.0        0.71        perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up
      0.68 ±  3%      -0.0        0.64 ±  3%  perf-profile.calltrace.cycles-pp.update_curr.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up
      1.61            -0.0        1.59        perf-profile.calltrace.cycles-pp.reweight_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.autoremove_wake_function
      1.10            +0.0        1.12        perf-profile.calltrace.cycles-pp.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common
      1.19            +0.0        1.23        perf-profile.calltrace.cycles-pp.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
      0.62 ±  4%      +0.1        0.68 ±  2%  perf-profile.calltrace.cycles-pp.file_update_time.pipe_write.__vfs_write.vfs_write.ksys_write
      1.23 ±  3%      +0.1        1.29        perf-profile.calltrace.cycles-pp.mutex_lock.pipe_write.__vfs_write.vfs_write.ksys_write
      1.28 ±  2%      +0.1        1.35        perf-profile.calltrace.cycles-pp.update_load_avg.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up
      0.60 ±  3%      +0.1        0.67 ±  3%  perf-profile.calltrace.cycles-pp.file_has_perm.security_file_permission.vfs_write.ksys_write.do_syscall_64
      1.64 ±  2%      +0.1        1.71 ±  2%  perf-profile.calltrace.cycles-pp.copyout.copy_page_to_iter.pipe_read.__vfs_read.vfs_read
      1.15 ±  4%      +0.1        1.24 ±  2%  perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_write.ksys_write.do_syscall_64
      0.66 ±  7%      +0.1        0.76 ±  5%  perf-profile.calltrace.cycles-pp.file_has_perm.security_file_permission.vfs_read.ksys_read.do_syscall_64
      2.02 ±  2%      +0.1        2.14 ±  2%  perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.__vfs_write.vfs_write.ksys_write
      1.06 ±  2%      +0.1        1.19 ±  5%  perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.__vfs_write
      0.71            +0.2        0.86 ±  5%  perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule.pipe_wait.pipe_read
      1.38 ±  2%      +0.2        1.54 ±  5%  perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.__vfs_write.vfs_write
      1.82 ±  4%      +0.2        1.99 ±  2%  perf-profile.calltrace.cycles-pp.security_file_permission.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.54            +0.2        0.71 ±  6%  perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.__schedule.schedule.pipe_wait
      0.41 ± 57%      +0.2        0.59 ±  3%  perf-profile.calltrace.cycles-pp.mutex_lock.pipe_read.__vfs_read.vfs_read.ksys_read
      2.86 ±  2%      +0.2        3.07        perf-profile.calltrace.cycles-pp.copy_page_to_iter.pipe_read.__vfs_read.vfs_read.ksys_read
      1.91 ±  2%      +0.2        2.13        perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_write.__vfs_write.vfs_write.ksys_write
      3.12 ±  2%      +0.4        3.50 ±  3%  perf-profile.calltrace.cycles-pp._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
      2.64 ±  3%      +0.4        3.03 ±  4%  perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common
      0.13 ±173%      +0.4        0.56 ±  3%  perf-profile.calltrace.cycles-pp.__mutex_unlock_slowpath.pipe_write.__vfs_write.vfs_write.ksys_write
      0.00            +0.6        0.56 ±  4%  perf-profile.calltrace.cycles-pp.__mutex_lock.pipe_write.__vfs_write.vfs_write.ksys_write
      5.25 ±  3%      +0.7        5.92        perf-profile.calltrace.cycles-pp.syscall_return_via_sysret
      6.20 ± 12%      -2.0        4.20 ±  2%  perf-profile.children.cycles-pp.select_task_rq_fair
     23.05 ±  3%      -1.9       21.13        perf-profile.children.cycles-pp.autoremove_wake_function
     23.73 ±  3%      -1.9       21.85        perf-profile.children.cycles-pp.__wake_up_common
      4.36 ± 17%      -1.9        2.48        perf-profile.children.cycles-pp.select_idle_sibling
     23.06 ±  3%      -1.8       21.22        perf-profile.children.cycles-pp.try_to_wake_up
     26.51 ±  2%      -1.7       24.78        perf-profile.children.cycles-pp.__wake_up_common_lock
      2.55 ± 23%      -1.6        0.96        perf-profile.children.cycles-pp.available_idle_cpu
     23.81            -1.1       22.68        perf-profile.children.cycles-pp.schedule
     79.38            -1.1       78.29        perf-profile.children.cycles-pp.do_syscall_64
     23.80            -1.1       22.73        perf-profile.children.cycles-pp.__schedule
     80.06            -1.0       79.06        perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
      6.98 ±  2%      -0.9        6.06 ±  2%  perf-profile.children.cycles-pp.exit_to_usermode_loop
      5.43            -0.4        5.01 ±  2%  perf-profile.children.cycles-pp.switch_mm_irqs_off
      0.65 ± 21%      -0.4        0.27 ±  4%  perf-profile.children.cycles-pp.cpumask_next_wrap
      3.81 ±  2%      -0.3        3.52 ±  2%  perf-profile.children.cycles-pp.update_curr
      1.78 ±  5%      -0.3        1.53 ±  4%  perf-profile.children.cycles-pp.___perf_sw_event
      1.32 ±  6%      -0.2        1.08 ±  5%  perf-profile.children.cycles-pp.set_task_cpu
      2.54            -0.2        2.31        perf-profile.children.cycles-pp.__switch_to
      0.50 ± 10%      -0.2        0.31 ±  3%  perf-profile.children.cycles-pp.find_next_bit
      1.12 ±  6%      -0.2        0.94        perf-profile.children.cycles-pp.update_rq_clock
      4.11 ±  2%      -0.2        3.95        perf-profile.children.cycles-pp.pick_next_task_fair
      0.64 ±  5%      -0.1        0.49 ±  7%  perf-profile.children.cycles-pp.migrate_task_rq_fair
      1.64            -0.1        1.51        perf-profile.children.cycles-pp.native_write_msr
      2.92            -0.1        2.79        perf-profile.children.cycles-pp.load_new_mm_cr3
      0.57 ±  7%      -0.1        0.47 ±  7%  perf-profile.children.cycles-pp.update_min_vruntime
      1.10 ±  2%      -0.1        1.01        perf-profile.children.cycles-pp.finish_task_switch
      0.96 ±  2%      -0.1        0.88 ±  2%  perf-profile.children.cycles-pp.__calc_delta
      0.33 ±  2%      -0.1        0.25 ±  5%  perf-profile.children.cycles-pp.remove_entity_load_avg
      0.88 ±  2%      -0.1        0.82        perf-profile.children.cycles-pp.put_prev_entity
      0.55 ±  6%      -0.1        0.49 ±  4%  perf-profile.children.cycles-pp.rb_erase_cached
      0.29 ±  7%      -0.1        0.23 ±  3%  perf-profile.children.cycles-pp.__switch_to_asm
      0.32 ±  3%      -0.1        0.26 ±  2%  perf-profile.children.cycles-pp.update_cfs_rq_h_load
      0.27 ±  2%      -0.0        0.24 ±  5%  perf-profile.children.cycles-pp.cpumask_next
      0.36            -0.0        0.33 ±  2%  perf-profile.children.cycles-pp.activate_task
      0.16 ±  6%      -0.0        0.14 ±  7%  perf-profile.children.cycles-pp.attach_entity_load_avg
      0.17 ±  4%      +0.0        0.19 ±  4%  perf-profile.children.cycles-pp.native_irq_return_iret
      0.11 ±  9%      +0.0        0.14 ±  5%  perf-profile.children.cycles-pp.mutex_spin_on_owner
      0.12 ±  8%      +0.0        0.15 ±  2%  perf-profile.children.cycles-pp.smp_reschedule_interrupt
      0.04 ± 57%      +0.0        0.07 ±  7%  perf-profile.children.cycles-pp.schedule_preempt_disabled
      0.16 ± 11%      +0.0        0.19 ±  6%  perf-profile.children.cycles-pp.timespec_trunc
      0.12 ± 12%      +0.0        0.15 ±  7%  perf-profile.children.cycles-pp.inode_has_perm
      0.30 ±  2%      +0.0        0.33 ±  5%  perf-profile.children.cycles-pp.__x64_sys_read
      1.11            +0.0        1.14        perf-profile.children.cycles-pp.check_preempt_curr
      0.29            +0.0        0.33 ±  5%  perf-profile.children.cycles-pp.generic_pipe_buf_confirm
      0.22 ±  4%      +0.0        0.26 ±  2%  perf-profile.children.cycles-pp.__fsnotify_parent
      0.30 ±  6%      +0.0        0.34 ±  6%  perf-profile.children.cycles-pp.rw_verify_area
      0.23 ±  3%      +0.0        0.28 ±  7%  perf-profile.children.cycles-pp.__x64_sys_write
      0.23 ±  6%      +0.0        0.28        perf-profile.children.cycles-pp.current_kernel_time64
      1.21            +0.0        1.26        perf-profile.children.cycles-pp.ttwu_do_wakeup
      0.45            +0.1        0.50 ±  2%  perf-profile.children.cycles-pp.entry_SYSCALL_64_stage2
      1.13            +0.1        1.18 ±  2%  perf-profile.children.cycles-pp.__update_load_avg_se
      0.00            +0.1        0.06 ±  7%  perf-profile.children.cycles-pp.osq_lock
      0.00            +0.1        0.06 ± 11%  perf-profile.children.cycles-pp.schedule_idle
      0.18 ±  4%      +0.1        0.24 ± 10%  perf-profile.children.cycles-pp.reschedule_interrupt
      0.00            +0.1        0.07 ± 36%  perf-profile.children.cycles-pp.scheduler_ipi
      0.41 ±  6%      +0.1        0.48 ±  2%  perf-profile.children.cycles-pp.wake_up_q
      0.26 ±  4%      +0.1        0.34 ±  5%  perf-profile.children.cycles-pp.preempt_schedule_common
      0.23 ±  8%      +0.1        0.30 ±  2%  perf-profile.children.cycles-pp.rcu_all_qs
      1.65            +0.1        1.73        perf-profile.children.cycles-pp.copyout
      0.42 ±  4%      +0.1        0.50        perf-profile.children.cycles-pp.current_time
      0.42 ±  5%      +0.1        0.51 ±  3%  perf-profile.children.cycles-pp.__might_fault
      0.73 ±  3%      +0.1        0.84        perf-profile.children.cycles-pp.__might_sleep
      0.00            +0.1        0.11 ± 13%  perf-profile.children.cycles-pp.__bitmap_weight
      0.00            +0.1        0.12 ±  3%  perf-profile.children.cycles-pp.intel_idle
      0.80 ±  6%      +0.1        0.93 ±  2%  perf-profile.children.cycles-pp.fsnotify
      0.00            +0.1        0.13        perf-profile.children.cycles-pp.cpuidle_enter_state
      1.09 ±  5%      +0.1        1.23 ±  3%  perf-profile.children.cycles-pp.avc_has_perm
      2.15 ±  2%      +0.1        2.29 ±  2%  perf-profile.children.cycles-pp.copy_page_from_iter
      0.67 ±  4%      +0.1        0.82        perf-profile.children.cycles-pp._cond_resched
      0.66 ±  4%      +0.2        0.83 ±  3%  perf-profile.children.cycles-pp.__mutex_lock
      2.48 ±  2%      +0.2        2.65        perf-profile.children.cycles-pp.mutex_lock
      0.95 ±  6%      +0.2        1.12        perf-profile.children.cycles-pp.__mutex_unlock_slowpath
      3.09 ±  2%      +0.2        3.27        perf-profile.children.cycles-pp.copy_page_to_iter
      0.04 ± 58%      +0.2        0.23 ±  3%  perf-profile.children.cycles-pp.secondary_startup_64
      0.04 ± 58%      +0.2        0.23 ±  3%  perf-profile.children.cycles-pp.start_secondary
      0.04 ± 58%      +0.2        0.23 ±  3%  perf-profile.children.cycles-pp.cpu_startup_entry
      0.04 ± 58%      +0.2        0.23 ±  3%  perf-profile.children.cycles-pp.do_idle
      1.10 ±  3%      +0.2        1.29 ±  3%  perf-profile.children.cycles-pp.___might_sleep
      0.76 ±  3%      +0.2        0.96 ±  2%  perf-profile.children.cycles-pp.__inode_security_revalidate
      1.40 ±  5%      +0.2        1.64 ±  4%  perf-profile.children.cycles-pp.file_has_perm
      2.76 ±  2%      +0.3        3.04 ±  3%  perf-profile.children.cycles-pp.mutex_unlock
      4.95            +0.5        5.41 ±  3%  perf-profile.children.cycles-pp._raw_spin_lock
      5.25 ±  3%      +0.7        5.92        perf-profile.children.cycles-pp.syscall_return_via_sysret
      5.63 ±  2%      +0.7        6.30 ±  3%  perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
      2.55 ± 23%      -1.6        0.96        perf-profile.self.cycles-pp.available_idle_cpu
      2.95 ±  3%      -0.3        2.65 ±  3%  perf-profile.self.cycles-pp.switch_mm_irqs_off
      1.69 ±  5%      -0.2        1.44 ±  4%  perf-profile.self.cycles-pp.___perf_sw_event
      0.39 ± 22%      -0.2        0.15 ±  4%  perf-profile.self.cycles-pp.cpumask_next_wrap
      2.54            -0.2        2.31        perf-profile.self.cycles-pp.__switch_to
      0.50 ± 10%      -0.2        0.31 ±  3%  perf-profile.self.cycles-pp.find_next_bit
      0.74 ± 10%      -0.2        0.55 ±  4%  perf-profile.self.cycles-pp.update_rq_clock
      2.77 ±  2%      -0.2        2.60 ±  3%  perf-profile.self.cycles-pp.__schedule
      1.64            -0.1        1.51        perf-profile.self.cycles-pp.native_write_msr
      2.92            -0.1        2.79        perf-profile.self.cycles-pp.load_new_mm_cr3
      2.32            -0.1        2.20 ±  3%  perf-profile.self.cycles-pp.update_curr
      0.57 ±  7%      -0.1        0.47 ±  7%  perf-profile.self.cycles-pp.update_min_vruntime
      1.20            -0.1        1.10 ±  2%  perf-profile.self.cycles-pp._raw_spin_lock
      0.96 ±  2%      -0.1        0.88 ±  2%  perf-profile.self.cycles-pp.__calc_delta
      0.26 ±  9%      -0.1        0.20 ±  7%  perf-profile.self.cycles-pp.migrate_task_rq_fair
      0.29 ±  7%      -0.1        0.23 ±  3%  perf-profile.self.cycles-pp.__switch_to_asm
      0.41 ±  2%      -0.1        0.36 ±  9%  perf-profile.self.cycles-pp.pipe_wait
      0.32 ±  3%      -0.1        0.26 ±  2%  perf-profile.self.cycles-pp.update_cfs_rq_h_load
      1.22            -0.1        1.17 ±  2%  perf-profile.self.cycles-pp.enqueue_task_fair
      0.33 ±  2%      -0.0        0.30 ±  2%  perf-profile.self.cycles-pp.prepare_to_wait
      0.36            -0.0        0.33 ±  2%  perf-profile.self.cycles-pp.activate_task
      0.16 ±  6%      -0.0        0.14 ±  7%  perf-profile.self.cycles-pp.attach_entity_load_avg
      0.05 ±  8%      +0.0        0.07 ±  6%  perf-profile.self.cycles-pp.cpumask_next
      0.09            +0.0        0.11 ±  4%  perf-profile.self.cycles-pp.wake_up_q
      0.21 ±  4%      +0.0        0.22 ±  4%  perf-profile.self.cycles-pp.ksys_write
      0.11 ±  7%      +0.0        0.13 ±  3%  perf-profile.self.cycles-pp.ttwu_do_wakeup
      0.23 ±  3%      +0.0        0.25        perf-profile.self.cycles-pp.check_preempt_curr
      0.12 ±  6%      +0.0        0.15 ±  3%  perf-profile.self.cycles-pp.__might_fault
      0.07 ± 10%      +0.0        0.10 ±  5%  perf-profile.self.cycles-pp.copyout
      0.17 ±  4%      +0.0        0.19 ±  4%  perf-profile.self.cycles-pp.native_irq_return_iret
      0.29            +0.0        0.32 ±  6%  perf-profile.self.cycles-pp.__x64_sys_read
      0.21 ±  2%      +0.0        0.24 ±  4%  perf-profile.self.cycles-pp.ksys_read
      0.12 ± 12%      +0.0        0.15 ±  7%  perf-profile.self.cycles-pp.inode_has_perm
      0.11 ±  7%      +0.0        0.14 ±  5%  perf-profile.self.cycles-pp.mutex_spin_on_owner
      0.29            +0.0        0.33 ±  5%  perf-profile.self.cycles-pp.generic_pipe_buf_confirm
      0.15 ± 10%      +0.0        0.19 ±  7%  perf-profile.self.cycles-pp.current_time
      0.27 ±  3%      +0.0        0.30 ±  7%  perf-profile.self.cycles-pp.schedule
      0.29            +0.0        0.33 ±  2%  perf-profile.self.cycles-pp._cond_resched
      0.15 ±  5%      +0.0        0.19 ±  2%  perf-profile.self.cycles-pp.__fdget_pos
      0.22 ±  5%      +0.0        0.26        perf-profile.self.cycles-pp.__fsnotify_parent
      0.22 ±  8%      +0.0        0.26        perf-profile.self.cycles-pp.wake_q_add
      0.23 ±  4%      +0.0        0.27        perf-profile.self.cycles-pp.current_kernel_time64
      0.22 ±  5%      +0.0        0.26 ±  4%  perf-profile.self.cycles-pp.__mutex_unlock_slowpath
      0.00            +0.1        0.05        perf-profile.self.cycles-pp.sched_clock
      0.45            +0.1        0.50 ±  2%  perf-profile.self.cycles-pp.entry_SYSCALL_64_stage2
      1.09            +0.1        1.15 ±  2%  perf-profile.self.cycles-pp.__update_load_avg_se
      0.22            +0.1        0.28 ±  7%  perf-profile.self.cycles-pp.__x64_sys_write
      0.61 ±  4%      +0.1        0.67 ±  2%  perf-profile.self.cycles-pp.vfs_write
      0.57            +0.1        0.63 ±  3%  perf-profile.self.cycles-pp.__vfs_write
      0.00            +0.1        0.06 ±  7%  perf-profile.self.cycles-pp.osq_lock
      1.14            +0.1        1.20        perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
      0.35 ±  4%      +0.1        0.41 ±  6%  perf-profile.self.cycles-pp.__mutex_lock
      0.34 ±  6%      +0.1        0.40 ±  7%  perf-profile.self.cycles-pp.security_file_permission
      0.25 ±  5%      +0.1        0.32 ±  4%  perf-profile.self.cycles-pp.__inode_security_revalidate
      0.23 ±  8%      +0.1        0.30 ±  3%  perf-profile.self.cycles-pp.rcu_all_qs
      0.50 ±  4%      +0.1        0.58 ±  3%  perf-profile.self.cycles-pp.__vfs_read
      0.68            +0.1        0.76 ±  3%  perf-profile.self.cycles-pp.copy_page_from_iter
      0.81 ±  2%      +0.1        0.91 ±  2%  perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
      0.72 ±  3%      +0.1        0.83        perf-profile.self.cycles-pp.__might_sleep
      0.29 ±  6%      +0.1        0.39 ±  6%  perf-profile.self.cycles-pp.file_has_perm
      0.00            +0.1        0.11 ± 13%  perf-profile.self.cycles-pp.__bitmap_weight
      0.79 ±  6%      +0.1        0.91 ±  3%  perf-profile.self.cycles-pp.fsnotify
      0.00            +0.1        0.12 ±  3%  perf-profile.self.cycles-pp.intel_idle
      1.08 ±  4%      +0.1        1.21 ±  3%  perf-profile.self.cycles-pp.avc_has_perm
      2.37 ±  2%      +0.1        2.50        perf-profile.self.cycles-pp.update_load_avg
      1.09 ±  3%      +0.2        1.27 ±  3%  perf-profile.self.cycles-pp.___might_sleep
      2.73 ±  2%      +0.3        2.99 ±  3%  perf-profile.self.cycles-pp.mutex_unlock
      1.19 ±  5%      +0.3        1.53 ±  7%  perf-profile.self.cycles-pp.do_syscall_64
      5.25 ±  3%      +0.7        5.92        perf-profile.self.cycles-pp.syscall_return_via_sysret
      5.63 ±  2%      +0.7        6.30 ±  3%  perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath





Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


Thanks,
Xiaolong

View attachment "config-4.17.0-rc6-00217-g8dd6626" of type "text/plain" (164361 bytes)

View attachment "job-script" of type "text/plain" (6980 bytes)

View attachment "job.yaml" of type "text/plain" (4654 bytes)

View attachment "reproduce" of type "text/plain" (3938 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ