lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [day] [month] [year] [list]
Message-ID: <20200507105403.GO5770@shao2-debian>
Date:   Thu, 7 May 2020 18:54:04 +0800
From:   kernel test robot <rong.a.chen@...el.com>
To:     Prathu Baronia <prathu.baronia@...plus.com>
Cc:     0day robot <lkp@...el.com>, LKML <linux-kernel@...r.kernel.org>,
        lkp@...ts.01.org
Subject: [mm] 9266baf8ee: vm-scalability.median -17.0% regression

Greeting,

FYI, we noticed a -17.0% regression of vm-scalability.median due to commit:


commit: 9266baf8eea588a78e5a0aa4ccd6cae36ff7349d ("mm: Optimized hugepage zeroing & copying from user")
url: https://github.com/0day-ci/linux/commits/UPDATE-20200501-184514/Prathu-Baronia/mm-Optimized-hugepage-zeroing-copying-from-user/20200415-041712


in testcase: vm-scalability
on test machine: 16 threads Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory
with following parameters:

	runtime: 300s
	size: 8T
	test: anon-w-seq-hugetlb
	cpufreq_governor: performance
	ucode: 0xca

test-description: The motivation behind this suite is to exercise functions and regions of the mm/ of the Linux kernel which are of interest to us.
test-url: https://git.kernel.org/cgit/linux/kernel/git/wfg/vm-scalability.git/

In addition to that, the commit also has significant impact on the following tests:

+------------------+----------------------------------------------------------------------+
| testcase: change | will-it-scale: will-it-scale.per_process_ops -4.9% regression        |
| test machine     | 4 threads Intel(R) Core(TM) i3-3220 CPU @ 3.30GHz with 8G memory     |
| test parameters  | cpufreq_governor=performance                                         |
|                  | mode=process                                                         |
|                  | nr_task=16                                                           |
|                  | test=page_fault1                                                     |
|                  | ucode=0x21                                                           |
+------------------+----------------------------------------------------------------------+
| testcase: change | vm-scalability: vm-scalability.median 8.4% improvement               |
| test machine     | 4 threads Intel(R) Core(TM) i3-3220 CPU @ 3.30GHz with 8G memory     |
| test parameters  | cpufreq_governor=performance                                         |
|                  | runtime=300s                                                         |
|                  | size=8T                                                              |
|                  | test=anon-w-seq-mt                                                   |
|                  | ucode=0x21                                                           |
+------------------+----------------------------------------------------------------------+
| testcase: change | vm-scalability: vm-scalability.median -2.6% regression               |
| test machine     | 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 256G memory |
| test parameters  | cpufreq_governor=performance                                         |
|                  | runtime=300s                                                         |
|                  | size=8T                                                              |
|                  | test=anon-cow-seq-hugetlb                                            |
|                  | ucode=0x500002c                                                      |
+------------------+----------------------------------------------------------------------+
| testcase: change | will-it-scale: will-it-scale.per_thread_ops -3.5% regression         |
| test machine     | 8 threads Intel(R) Core(TM) i7-3770K CPU @ 3.50GHz with 16G memory   |
| test parameters  | cpufreq_governor=performance                                         |
|                  | mode=thread                                                          |
|                  | nr_task=100%                                                         |
|                  | test=page_fault1                                                     |
|                  | ucode=0x21                                                           |
+------------------+----------------------------------------------------------------------+
| testcase: change | will-it-scale: will-it-scale.per_process_ops -5.3% regression        |
| test machine     | 4 threads Intel(R) Core(TM) i3-3220 CPU @ 3.30GHz with 8G memory     |
| test parameters  | cpufreq_governor=performance                                         |
|                  | test=page_fault1                                                     |
|                  | ucode=0x21                                                           |
+------------------+----------------------------------------------------------------------+


If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen@...el.com>


Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp install job.yaml  # job file is attached in this email
        bin/lkp run     job.yaml

=========================================================================================
compiler/cpufreq_governor/kconfig/rootfs/runtime/size/tbox_group/test/testcase/ucode:
  gcc-7/performance/x86_64-rhel-7.6/debian-x86_64-20191114.cgz/300s/8T/lkp-cfl-e1/anon-w-seq-hugetlb/vm-scalability/0xca

commit: 
  8632e9b564 (" hyperv-fixes for 5.7-rc1")
  9266baf8ee ("mm: Optimized hugepage zeroing & copying from user")

8632e9b5645bbc23 9266baf8eea588a78e5a0aa4ccd 
---------------- --------------------------- 
       fail:runs  %reproduction    fail:runs
           |             |             |    
           :4           25%           1:4     dmesg.WARNING:at#for_ip_interrupt_entry/0x
         %stddev     %change         %stddev
             \          |                \  
      0.00           +35.0%       0.00 ± 41%  vm-scalability.free_time
    885731           -17.0%     735379        vm-scalability.median
  14052443           -16.6%   11724633        vm-scalability.throughput
    130581 ±  2%      -9.0%     118824 ±  3%  vm-scalability.time.involuntary_context_switches
   2059540           -15.5%    1739286        vm-scalability.time.minor_page_faults
      1466            +1.4%       1487        vm-scalability.time.percent_of_cpu_this_job_got
      1430           +17.8%       1685        vm-scalability.time.system_time
      2993            -6.5%       2797        vm-scalability.time.user_time
     16555           -12.1%      14554        vm-scalability.time.voluntary_context_switches
 3.937e+09           -15.6%  3.323e+09        vm-scalability.workload
    322.62            -0.9%     319.64        boot-time.idle
      5.46 ±  2%      -1.0        4.50 ±  2%  mpstat.cpu.all.idle%
     31.64            +5.1       36.71        mpstat.cpu.all.sys%
     62.00            -6.5%      58.00        vmstat.cpu.us
      2220 ±  2%     -10.8%       1980        vmstat.system.cs
      1140 ±  5%     -12.6%     997.00 ±  7%  slabinfo.kmalloc-rcl-96.active_objs
      1140 ±  5%     -12.6%     997.00 ±  7%  slabinfo.kmalloc-rcl-96.num_objs
      6311            -9.9%       5685        slabinfo.vmap_area.active_objs
      6311            -9.9%       5685        slabinfo.vmap_area.num_objs
    132283 ±  9%     -12.8%     115384        softirqs.CPU0.TIMER
    137945 ±  7%     -12.9%     120092        softirqs.CPU4.TIMER
     26018 ±  4%     -11.9%      22933 ±  9%  softirqs.CPU5.RCU
    134170 ±  8%     -11.2%     119154        softirqs.CPU8.TIMER
    354273 ±  7%     -14.5%     303080 ±  7%  cpuidle.C1.time
   1473619 ± 37%     -29.0%    1045628 ±  2%  cpuidle.C1E.time
   4610710 ± 51%     -48.9%    2357842 ± 23%  cpuidle.C3.time
     23383 ± 41%     -39.8%      14070 ± 11%  cpuidle.C3.usage
 2.605e+08           -16.8%  2.166e+08        cpuidle.C6.time
    552177           -16.6%     460777        cpuidle.C6.usage
     16488 ±  9%     -30.4%      11472 ± 14%  cpuidle.POLL.usage
      8308 ±  5%     -11.4%       7364 ±  6%  interrupts.CAL:Function_call_interrupts
    479.00 ±  2%     -12.6%     418.50 ±  3%  interrupts.CPU0.CAL:Function_call_interrupts
    514.50 ±  4%     -13.0%     447.50 ±  9%  interrupts.CPU1.CAL:Function_call_interrupts
    529.50 ±  4%     -15.3%     448.75 ±  6%  interrupts.CPU10.CAL:Function_call_interrupts
    528.50 ±  4%     -13.0%     459.75 ±  6%  interrupts.CPU14.CAL:Function_call_interrupts
    526.25 ±  6%     -15.7%     443.50 ±  9%  interrupts.CPU2.CAL:Function_call_interrupts
    523.75 ±  3%      -9.6%     473.25 ±  6%  interrupts.CPU5.CAL:Function_call_interrupts
    518.75 ±  6%     -10.9%     462.25 ±  6%  interrupts.CPU6.CAL:Function_call_interrupts
      3608 ± 30%    +101.4%       7268 ± 20%  interrupts.CPU6.RES:Rescheduling_interrupts
     81528 ±  4%     -12.2%      71606 ±  5%  interrupts.RES:Rescheduling_interrupts
    373.33 ± 28%     +47.7%     551.38 ±  5%  sched_debug.cfs_rq:/.load_avg.max
     90.00 ± 34%     +50.7%     135.66        sched_debug.cfs_rq:/.load_avg.stddev
     13.32 ± 13%     +23.0%      16.39 ±  6%  sched_debug.cfs_rq:/.nr_spread_over.avg
      4816 ± 17%     -31.8%       3286 ± 22%  sched_debug.cpu.curr->pid.min
     39104 ±  3%     -10.7%      34937        sched_debug.cpu.nr_switches.max
      7502 ±  7%     -12.5%       6566 ±  3%  sched_debug.cpu.nr_switches.stddev
     34374 ±  5%     -14.3%      29462 ±  5%  sched_debug.cpu.sched_count.max
      6717 ± 12%     -16.4%       5617 ±  4%  sched_debug.cpu.sched_count.stddev
      2224 ±  5%     -13.4%       1925 ±  4%  sched_debug.cpu.sched_goidle.avg
      1468 ±  5%     -19.5%       1181 ±  3%  sched_debug.cpu.sched_goidle.min
     17663 ±  7%     -14.0%      15196 ±  4%  sched_debug.cpu.ttwu_count.max
      3887 ± 16%     -16.2%       3258 ±  4%  sched_debug.cpu.ttwu_count.stddev
     11677 ±  6%     -18.3%       9544 ±  9%  sched_debug.cpu.ttwu_local.max
      2596 ± 12%     -17.1%       2151 ±  6%  sched_debug.cpu.ttwu_local.stddev
   1709412           -15.6%    1442880        proc-vmstat.htlb_buddy_alloc_success
    167.50           -11.0%     149.00        proc-vmstat.nr_dirtied
    387471            +2.2%     395950        proc-vmstat.nr_dirty_background_threshold
    775891            +2.2%     792870        proc-vmstat.nr_dirty_threshold
   3943955            +2.2%    4030727        proc-vmstat.nr_free_pages
      8037            -2.4%       7847        proc-vmstat.nr_inactive_anon
      7418            -1.2%       7328        proc-vmstat.nr_mapped
     11491            -1.8%      11288        proc-vmstat.nr_shmem
    155.50           -11.3%     138.00        proc-vmstat.nr_written
      8037            -2.4%       7847        proc-vmstat.nr_zone_inactive_anon
   2351084           -13.1%    2042972        proc-vmstat.numa_hit
   2351084           -13.1%    2042972        proc-vmstat.numa_local
 8.759e+08           -15.6%  7.394e+08        proc-vmstat.pgalloc_normal
   2451008           -13.1%    2129664        proc-vmstat.pgfault
 8.749e+08           -15.7%  7.373e+08        proc-vmstat.pgfree
     25.20 ±  2%     -25.2        0.00        perf-profile.calltrace.cycles-pp.clear_subpage.clear_huge_page.hugetlb_no_page.hugetlb_fault.handle_mm_fault
     24.59 ±  2%     -24.6        0.00        perf-profile.calltrace.cycles-pp.clear_page_erms.clear_subpage.clear_huge_page.hugetlb_no_page.hugetlb_fault
     26.17            -4.7       21.52 ±  2%  perf-profile.calltrace.cycles-pp.do_rw_once
      3.15 ±  5%      -0.5        2.61 ±  5%  perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.alloc_fresh_huge_page.alloc_surplus_huge_page.hugetlb_acct_memory.hugetlb_reserve_pages
      3.15 ±  5%      -0.5        2.60 ±  5%  perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.alloc_fresh_huge_page.alloc_surplus_huge_page.hugetlb_acct_memory
      3.17 ±  5%      -0.5        2.63 ±  5%  perf-profile.calltrace.cycles-pp.alloc_fresh_huge_page.alloc_surplus_huge_page.hugetlb_acct_memory.hugetlb_reserve_pages.hugetlbfs_file_mmap
      3.20 ±  4%      -0.5        2.67 ±  5%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.mmap64
      3.20 ±  4%      -0.5        2.67 ±  5%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.mmap64
      3.20 ±  4%      -0.5        2.67 ±  5%  perf-profile.calltrace.cycles-pp.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe.mmap64
      3.20 ±  4%      -0.5        2.67 ±  5%  perf-profile.calltrace.cycles-pp.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe.mmap64
      3.20 ±  4%      -0.5        2.67 ±  5%  perf-profile.calltrace.cycles-pp.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe
      3.20 ±  4%      -0.5        2.67 ±  5%  perf-profile.calltrace.cycles-pp.mmap_region.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64
      3.20 ±  4%      -0.5        2.67 ±  5%  perf-profile.calltrace.cycles-pp.mmap64
      3.18 ±  5%      -0.5        2.64 ±  5%  perf-profile.calltrace.cycles-pp.alloc_surplus_huge_page.hugetlb_acct_memory.hugetlb_reserve_pages.hugetlbfs_file_mmap.mmap_region
      3.20 ±  4%      -0.5        2.66 ±  5%  perf-profile.calltrace.cycles-pp.hugetlb_acct_memory.hugetlb_reserve_pages.hugetlbfs_file_mmap.mmap_region.do_mmap
      3.20 ±  4%      -0.5        2.66 ±  5%  perf-profile.calltrace.cycles-pp.hugetlbfs_file_mmap.mmap_region.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff
      3.20 ±  4%      -0.5        2.66 ±  5%  perf-profile.calltrace.cycles-pp.hugetlb_reserve_pages.hugetlbfs_file_mmap.mmap_region.do_mmap.vm_mmap_pgoff
      2.79 ±  5%      -0.5        2.33 ±  5%  perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.get_page_from_freelist.__alloc_pages_nodemask.alloc_fresh_huge_page.alloc_surplus_huge_page
      2.78 ±  5%      -0.5        2.33 ±  5%  perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.get_page_from_freelist.__alloc_pages_nodemask.alloc_fresh_huge_page
      1.51 ±  8%      -0.3        1.18 ±  3%  perf-profile.calltrace.cycles-pp.ret_from_fork
      1.51 ±  8%      -0.3        1.18 ±  3%  perf-profile.calltrace.cycles-pp.kthread.ret_from_fork
      1.49 ±  8%      -0.3        1.17 ±  3%  perf-profile.calltrace.cycles-pp.drm_fb_helper_dirty_work.process_one_work.worker_thread.kthread.ret_from_fork
      1.50 ±  8%      -0.3        1.18 ±  3%  perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork
      1.50 ±  8%      -0.3        1.18 ±  3%  perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork
      1.45 ±  8%      -0.3        1.14 ±  3%  perf-profile.calltrace.cycles-pp.memcpy_erms.drm_fb_helper_dirty_work.process_one_work.worker_thread.kthread
      1.05 ±  5%      -0.3        0.79 ±  5%  perf-profile.calltrace.cycles-pp.console_unlock.vprintk_emit.devkmsg_emit.devkmsg_write.new_sync_write
      1.06 ±  5%      -0.3        0.80 ±  6%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write._fini
      1.06 ±  5%      -0.3        0.80 ±  6%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write._fini
      1.06 ±  5%      -0.3        0.80 ±  6%  perf-profile.calltrace.cycles-pp.vprintk_emit.devkmsg_emit.devkmsg_write.new_sync_write.vfs_write
      1.06 ±  5%      -0.3        0.80 ±  6%  perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write._fini
      1.06 ±  5%      -0.3        0.80 ±  6%  perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
      1.06 ±  5%      -0.3        0.80 ±  6%  perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.06 ±  5%      -0.3        0.80 ±  6%  perf-profile.calltrace.cycles-pp.write._fini
      1.06 ±  5%      -0.3        0.80 ±  6%  perf-profile.calltrace.cycles-pp.devkmsg_write.new_sync_write.vfs_write.ksys_write.do_syscall_64
      1.06 ±  5%      -0.3        0.80 ±  6%  perf-profile.calltrace.cycles-pp._fini
      1.06 ±  5%      -0.3        0.80 ±  6%  perf-profile.calltrace.cycles-pp.devkmsg_emit.devkmsg_write.new_sync_write.vfs_write.ksys_write
      0.89 ±  6%      -0.2        0.66 ±  5%  perf-profile.calltrace.cycles-pp.serial8250_console_write.console_unlock.vprintk_emit.devkmsg_emit.devkmsg_write
      0.83 ±  7%      -0.2        0.60 ±  5%  perf-profile.calltrace.cycles-pp.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit.devkmsg_emit
      1.42 ±  3%      -0.2        1.24 ±  6%  perf-profile.calltrace.cycles-pp.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.42 ±  3%      -0.2        1.24 ±  6%  perf-profile.calltrace.cycles-pp.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.42 ±  3%      -0.2        1.24 ±  6%  perf-profile.calltrace.cycles-pp.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.41 ±  2%      -0.2        1.23 ±  6%  perf-profile.calltrace.cycles-pp.mmput.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
      1.41 ±  2%      -0.2        1.23 ±  6%  perf-profile.calltrace.cycles-pp.exit_mmap.mmput.do_exit.do_group_exit.__x64_sys_exit_group
      1.39 ±  2%      -0.2        1.21 ±  6%  perf-profile.calltrace.cycles-pp.tlb_finish_mmu.exit_mmap.mmput.do_exit.do_group_exit
      1.39 ±  2%      -0.2        1.21 ±  6%  perf-profile.calltrace.cycles-pp.tlb_flush_mmu.tlb_finish_mmu.exit_mmap.mmput.do_exit
      1.39 ±  2%      -0.2        1.21 ±  6%  perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu.tlb_finish_mmu.exit_mmap.mmput
      1.38 ±  2%      -0.2        1.20 ±  6%  perf-profile.calltrace.cycles-pp.__free_huge_page.release_pages.tlb_flush_mmu.tlb_finish_mmu.exit_mmap
      1.44 ±  2%      -0.2        1.27 ±  5%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
      1.44 ±  2%      -0.2        1.27 ±  5%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.05 ±  3%      -0.2        0.88 ±  8%  perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.__free_huge_page.release_pages.tlb_flush_mmu
      1.05 ±  3%      -0.2        0.88 ±  8%  perf-profile.calltrace.cycles-pp._raw_spin_lock.__free_huge_page.release_pages.tlb_flush_mmu.tlb_finish_mmu
      0.78 ±  7%      -0.1        0.66 ±  4%  perf-profile.calltrace.cycles-pp.secondary_startup_64
      0.74 ±  8%      -0.1        0.62 ±  6%  perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
      0.75 ±  8%      -0.1        0.63 ±  6%  perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64
      0.75 ±  8%      -0.1        0.63 ±  6%  perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
      0.75 ±  8%      -0.1        0.63 ±  6%  perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64
      0.74 ±  8%      -0.1        0.62 ±  6%  perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
      0.00            +0.9        0.95        perf-profile.calltrace.cycles-pp.mutex_spin_on_owner.__mutex_lock.hugetlb_fault.handle_mm_fault.do_page_fault
      0.00            +1.0        0.97        perf-profile.calltrace.cycles-pp.__mutex_lock.hugetlb_fault.handle_mm_fault.do_page_fault.page_fault
     27.03 ±  2%      +5.1       32.11 ±  4%  perf-profile.calltrace.cycles-pp.hugetlb_no_page.hugetlb_fault.handle_mm_fault.do_page_fault.page_fault
     26.48 ±  2%      +5.3       31.80 ±  4%  perf-profile.calltrace.cycles-pp.clear_huge_page.hugetlb_no_page.hugetlb_fault.handle_mm_fault.do_page_fault
     25.54            +5.8       31.33 ±  2%  perf-profile.calltrace.cycles-pp.page_fault.do_access
     25.48            +5.8       31.28 ±  2%  perf-profile.calltrace.cycles-pp.do_page_fault.page_fault.do_access
     25.41            +5.8       31.22 ±  2%  perf-profile.calltrace.cycles-pp.handle_mm_fault.do_page_fault.page_fault.do_access
     25.38            +5.8       31.19 ±  2%  perf-profile.calltrace.cycles-pp.hugetlb_fault.handle_mm_fault.do_page_fault.page_fault.do_access
     89.02            +9.2       98.24 ±  2%  perf-profile.calltrace.cycles-pp.do_access
     25.46 ±  2%     -25.5        0.00        perf-profile.children.cycles-pp.clear_subpage
     24.72 ±  2%     -24.7        0.00        perf-profile.children.cycles-pp.clear_page_erms
      5.90 ±  2%      -1.0        4.92 ±  2%  perf-profile.children.cycles-pp.do_syscall_64
      5.90 ±  2%      -1.0        4.93 ±  3%  perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
      0.73 ±  3%      -0.7        0.06 ±  7%  perf-profile.children.cycles-pp.___might_sleep
      3.91 ±  3%      -0.6        3.29 ±  4%  perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
      3.20 ±  4%      -0.5        2.67 ±  5%  perf-profile.children.cycles-pp.mmap64
      3.17 ±  5%      -0.5        2.64 ±  5%  perf-profile.children.cycles-pp.alloc_fresh_huge_page
      3.16 ±  5%      -0.5        2.62 ±  5%  perf-profile.children.cycles-pp.get_page_from_freelist
      3.21 ±  4%      -0.5        2.68 ±  5%  perf-profile.children.cycles-pp.vm_mmap_pgoff
      3.18 ±  5%      -0.5        2.65 ±  5%  perf-profile.children.cycles-pp.alloc_surplus_huge_page
      3.20 ±  4%      -0.5        2.67 ±  5%  perf-profile.children.cycles-pp.hugetlbfs_file_mmap
      3.20 ±  4%      -0.5        2.67 ±  5%  perf-profile.children.cycles-pp.hugetlb_reserve_pages
      3.20 ±  4%      -0.5        2.68 ±  5%  perf-profile.children.cycles-pp.ksys_mmap_pgoff
      3.20 ±  4%      -0.5        2.68 ±  5%  perf-profile.children.cycles-pp.do_mmap
      3.20 ±  4%      -0.5        2.68 ±  5%  perf-profile.children.cycles-pp.mmap_region
      3.16 ±  5%      -0.5        2.63 ±  5%  perf-profile.children.cycles-pp.__alloc_pages_nodemask
      3.20 ±  4%      -0.5        2.68 ±  5%  perf-profile.children.cycles-pp.hugetlb_acct_memory
      2.84 ±  4%      -0.5        2.35 ±  5%  perf-profile.children.cycles-pp._raw_spin_lock_irqsave
      1.51 ±  8%      -0.3        1.18 ±  3%  perf-profile.children.cycles-pp.kthread
      1.49 ±  8%      -0.3        1.17 ±  3%  perf-profile.children.cycles-pp.memcpy_erms
      1.49 ±  8%      -0.3        1.17 ±  3%  perf-profile.children.cycles-pp.drm_fb_helper_dirty_work
      1.51 ±  8%      -0.3        1.19 ±  3%  perf-profile.children.cycles-pp.ret_from_fork
      1.50 ±  8%      -0.3        1.18 ±  3%  perf-profile.children.cycles-pp.process_one_work
      1.50 ±  8%      -0.3        1.18 ±  3%  perf-profile.children.cycles-pp.worker_thread
      1.18 ±  4%      -0.3        0.89 ±  2%  perf-profile.children.cycles-pp.console_unlock
      1.01 ±  5%      -0.3        0.75        perf-profile.children.cycles-pp.serial8250_console_write
      1.19 ±  4%      -0.3        0.92 ±  3%  perf-profile.children.cycles-pp.vprintk_emit
      1.08 ±  4%      -0.3        0.83 ±  6%  perf-profile.children.cycles-pp.write
      1.09 ±  5%      -0.3        0.84 ±  7%  perf-profile.children.cycles-pp.new_sync_write
      1.10 ±  5%      -0.3        0.85 ±  7%  perf-profile.children.cycles-pp.ksys_write
      1.10 ±  5%      -0.3        0.85 ±  7%  perf-profile.children.cycles-pp.vfs_write
      1.06 ±  5%      -0.3        0.80 ±  6%  perf-profile.children.cycles-pp.devkmsg_write
      1.06 ±  5%      -0.3        0.80 ±  6%  perf-profile.children.cycles-pp._fini
      1.06 ±  5%      -0.3        0.80 ±  6%  perf-profile.children.cycles-pp.devkmsg_emit
      0.94 ±  6%      -0.2        0.69        perf-profile.children.cycles-pp.uart_console_write
      0.81 ±  5%      -0.2        0.60 ±  2%  perf-profile.children.cycles-pp.wait_for_xmitr
      0.79 ±  5%      -0.2        0.58 ±  2%  perf-profile.children.cycles-pp.io_serial_in
      0.75 ±  5%      -0.2        0.55        perf-profile.children.cycles-pp.serial8250_console_putchar
      1.43 ±  2%      -0.2        1.25 ±  6%  perf-profile.children.cycles-pp.__x64_sys_exit_group
      1.43 ±  2%      -0.2        1.25 ±  6%  perf-profile.children.cycles-pp.do_group_exit
      1.43 ±  2%      -0.2        1.25 ±  6%  perf-profile.children.cycles-pp.do_exit
      1.39 ±  2%      -0.2        1.21 ±  6%  perf-profile.children.cycles-pp.tlb_finish_mmu
      1.39 ±  2%      -0.2        1.21 ±  6%  perf-profile.children.cycles-pp.tlb_flush_mmu
      1.42 ±  2%      -0.2        1.25 ±  6%  perf-profile.children.cycles-pp.mmput
      1.42 ±  2%      -0.2        1.25 ±  6%  perf-profile.children.cycles-pp.exit_mmap
      1.39 ±  2%      -0.2        1.21 ±  6%  perf-profile.children.cycles-pp.release_pages
      1.38 ±  2%      -0.2        1.20 ±  6%  perf-profile.children.cycles-pp.__free_huge_page
      1.19 ±  3%      -0.2        1.02 ±  7%  perf-profile.children.cycles-pp._raw_spin_lock
      0.78 ±  7%      -0.1        0.65 ±  4%  perf-profile.children.cycles-pp.cpuidle_enter
      0.78 ±  7%      -0.1        0.65 ±  4%  perf-profile.children.cycles-pp.cpuidle_enter_state
      0.78 ±  7%      -0.1        0.66 ±  4%  perf-profile.children.cycles-pp.secondary_startup_64
      0.78 ±  7%      -0.1        0.66 ±  4%  perf-profile.children.cycles-pp.cpu_startup_entry
      0.78 ±  7%      -0.1        0.66 ±  4%  perf-profile.children.cycles-pp.do_idle
      0.75 ±  8%      -0.1        0.63 ±  6%  perf-profile.children.cycles-pp.start_secondary
      0.74 ±  7%      -0.1        0.63 ±  4%  perf-profile.children.cycles-pp.intel_idle
      0.20 ±  8%      -0.1        0.15 ±  7%  perf-profile.children.cycles-pp.io_serial_out
      0.48 ±  5%      -0.1        0.43 ±  5%  perf-profile.children.cycles-pp.tick_sched_handle
      0.46 ±  6%      -0.0        0.42 ±  6%  perf-profile.children.cycles-pp.update_process_times
      0.15 ±  7%      -0.0        0.12 ±  5%  perf-profile.children.cycles-pp.alloc_huge_page
      0.08 ±  5%      -0.0        0.07        perf-profile.children.cycles-pp.update_curr
      0.43 ±  4%      +0.6        1.01 ±  8%  perf-profile.children.cycles-pp.mutex_spin_on_owner
      0.44 ±  5%      +0.6        1.04 ±  7%  perf-profile.children.cycles-pp.__mutex_lock
     27.03 ±  2%      +5.1       32.17 ±  4%  perf-profile.children.cycles-pp.hugetlb_no_page
     26.78 ±  2%      +5.2       31.95 ±  4%  perf-profile.children.cycles-pp.clear_huge_page
     27.75 ±  2%      +5.7       33.48 ±  4%  perf-profile.children.cycles-pp.page_fault
     27.68 ±  2%      +5.7       33.42 ±  4%  perf-profile.children.cycles-pp.do_page_fault
     27.61 ±  2%      +5.7       33.35 ±  4%  perf-profile.children.cycles-pp.handle_mm_fault
     27.54 ±  2%      +5.7       33.28 ±  4%  perf-profile.children.cycles-pp.hugetlb_fault
     64.78            +5.9       70.72 ±  2%  perf-profile.children.cycles-pp.do_access
     24.19 ±  2%     -24.2        0.00        perf-profile.self.cycles-pp.clear_page_erms
     13.76            -3.1       10.71 ±  2%  perf-profile.self.cycles-pp.do_access
      3.91 ±  3%      -0.6        3.28 ±  4%  perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
      0.46 ±  4%      -0.4        0.04 ± 58%  perf-profile.self.cycles-pp.___might_sleep
      1.47 ±  8%      -0.3        1.16 ±  3%  perf-profile.self.cycles-pp.memcpy_erms
      0.78 ±  4%      -0.2        0.57 ±  2%  perf-profile.self.cycles-pp.io_serial_in
      0.74 ±  7%      -0.1        0.63 ±  4%  perf-profile.self.cycles-pp.intel_idle
      0.30 ±  6%      -0.1        0.24 ±  3%  perf-profile.self.cycles-pp.get_page_from_freelist
      0.20 ±  8%      -0.1        0.15 ±  7%  perf-profile.self.cycles-pp.io_serial_out
      0.43 ±  5%      +0.6        1.00 ±  8%  perf-profile.self.cycles-pp.mutex_spin_on_owner
      0.32 ±  2%     +31.2       31.57 ±  4%  perf-profile.self.cycles-pp.clear_huge_page


                                                                                
                           vm-scalability.time.user_time                        
                                                                                
  3050 +--------------------------------------------------------------------+   
       |                                                                    |   
  3000 |-+         .+..    .+..    .+..  .+.    .+..    .+..+.. .+..    .+..|   
       |..+.+..+..+    +..+    +..+    +.   +..+    +..+       +    +..+    |   
       |                                                                    |   
  2950 |-+                                                                  |   
       |                                                                    |   
  2900 |-+                                                                  |   
       |                                                                    |   
  2850 |-+                                                                  |   
       |                                                                    |   
       |            O  O       O  O                                         |   
  2800 |-+O O  O  O       O O       O  O  O O  O O  O  O O  O  O O  O  O    |   
       |                                                                    |   
  2750 +--------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                          vm-scalability.time.system_time                       
                                                                                
  1750 +--------------------------------------------------------------------+   
       |                                                                    |   
  1700 |-+O O  O  O O  O  O O  O  O              O       O  O       O  O    |   
       |                            O  O  O O  O    O  O       O O          |   
  1650 |-+                                                                  |   
       |                                                                    |   
  1600 |-+                                                                  |   
       |                                                                    |   
  1550 |-+                                                                  |   
       |                                                                    |   
  1500 |-+                                                                  |   
       |                                                                    |   
  1450 |..+.+..+..+.  .+..+.             .+.+..+.+..+..+.     .+.       .+..|   
       |            +.      +..+..+.+..+.                +..+.   +..+..+    |   
  1400 +--------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                  vm-scalability.time.percent_of_cpu_this_job_got               
                                                                                
  1490 +--------------------------------------------------------------------+   
       |       O    O  O          O              O       O          O       |   
       |  O       O       O O             O O  O    O  O    O  O O     O    |   
  1485 |-+                          O  O                                    |   
       |                                                                    |   
       |                                                                    |   
  1480 |-+                                                                  |   
       |                                                                    |   
  1475 |-+                                                                  |   
       |                                                                    |   
       |                                                                    |   
  1470 |-+       .+                       +.    .+..          .+.        +  |   
       |.. .+..+.  +     .+        .+.. ..  +..+    +..+.  .+.   +..    : + |   
       |  +         +..+.  +     .+    +                 +.         +.. :  +|   
  1465 +--------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                         vm-scalability.time.minor_page_faults                  
                                                                                
   2.1e+06 +----------------------------------------------------------------+   
           |..+.+..+.+..+.+..+.+..+.+..+.+..+.+..+.+..+.+..+.+..+.+.. .+..+.|   
  2.05e+06 |-+                                                       +      |   
           |                                                                |   
     2e+06 |-+                                                              |   
  1.95e+06 |-+                                                              |   
           |                                                                |   
   1.9e+06 |-+                                                              |   
           |                                                                |   
  1.85e+06 |-+                                                              |   
   1.8e+06 |-+                                                              |   
           |                                                                |   
  1.75e+06 |-+                                     O                        |   
           |  O O  O O  O O  O O  O O  O O  O O  O    O O  O O  O O  O O    |   
   1.7e+06 +----------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                   vm-scalability.time.voluntary_context_switches               
                                                                                
  17000 +-------------------------------------------------------------------+   
        |            +.. .+..+..    .+.  .+..+.+..+.+..+..+.  .+..          |   
  16500 |..+.+..+. ..   +       +.+.   +.                   +.    +.+..+.+..|   
        |         +                                                         |   
        |                                                                   |   
  16000 |-+                                                                 |   
        |                                                                   |   
  15500 |-+                                                                 |   
        |                                                                   |   
  15000 |-+             O                                                   |   
        |                                 O  O O                            |   
        |                    O  O      O          O    O  O       O O       |   
  14500 |-+O O  O O  O    O       O  O              O       O  O       O    |   
        |                                                                   |   
  14000 +-------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                               vm-scalability.throughput                        
                                                                                
  1.45e+07 +----------------------------------------------------------------+   
           |                                                                |   
   1.4e+07 |..+.+..+.+..+.+..+.+..+.+..+.+..+.+..+.+..+.+..+.+..+.+..+.+..+.|   
           |                                                                |   
           |                                                                |   
  1.35e+07 |-+                                                              |   
           |                                                                |   
   1.3e+07 |-+                                                              |   
           |                                                                |   
  1.25e+07 |-+                                                              |   
           |                                                                |   
           |                                                                |   
   1.2e+07 |-+                                                              |   
           |  O O  O O  O O  O O  O O  O O  O O  O O  O O  O O  O O  O O    |   
  1.15e+07 +----------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                vm-scalability.median                           
                                                                                
  900000 +------------------------------------------------------------------+   
         |..+.+..+.+..+.+..+..+.+..+.+..+..+.+..+.+..+.+..+..+.+..+.+..+.+..|   
  880000 |-+                                                                |   
  860000 |-+                                                                |   
         |                                                                  |   
  840000 |-+                                                                |   
  820000 |-+                                                                |   
         |                                                                  |   
  800000 |-+                                                                |   
  780000 |-+                                                                |   
         |                                                                  |   
  760000 |-+                                                                |   
  740000 |-+                                                                |   
         |  O O  O O  O O  O  O O  O O  O  O O  O O  O O  O  O O  O O  O    |   
  720000 +------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                vm-scalability.workload                         
                                                                                
    4e+09 +-----------------------------------------------------------------+   
          |..+.+..+.+..+.+..+.+..+.+..+.+..+..+.+..+.+..+.+..+.+..+.+..+.+..|   
  3.9e+09 |-+                                                               |   
          |                                                                 |   
  3.8e+09 |-+                                                               |   
          |                                                                 |   
  3.7e+09 |-+                                                               |   
          |                                                                 |   
  3.6e+09 |-+                                                               |   
          |                                                                 |   
  3.5e+09 |-+                                                               |   
          |                                                                 |   
  3.4e+09 |-+                                                               |   
          |                                        O                        |   
  3.3e+09 +-----------------------------------------------------------------+   
                                                                                
                                                                                
[*] bisect-good sample
[O] bisect-bad  sample

***************************************************************************************************
lkp-ivb-d02: 4 threads Intel(R) Core(TM) i3-3220 CPU @ 3.30GHz with 8G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode:
  gcc-7/performance/x86_64-rhel-7.6/process/16/debian-x86_64-20191114.cgz/lkp-ivb-d02/page_fault1/will-it-scale/0x21

commit: 
  8632e9b564 (" hyperv-fixes for 5.7-rc1")
  9266baf8ee ("mm: Optimized hugepage zeroing & copying from user")

8632e9b5645bbc23 9266baf8eea588a78e5a0aa4ccd 
---------------- --------------------------- 
       fail:runs  %reproduction    fail:runs
           |             |             |    
          1:4            0%           1:4     perf-profile.children.cycles-pp.error_entry
         %stddev     %change         %stddev
             \          |                \  
    238214            -4.9%     226603        will-it-scale.per_process_ops
   3811432            -4.9%    3625669        will-it-scale.workload
     25486 ±  8%     +16.8%      29758 ±  7%  softirqs.CPU3.RCU
     20.96 ±  2%      +5.5%      22.11 ±  2%  boot-time.boot
     55.42 ±  2%      +6.0%      58.77 ±  2%  boot-time.idle
      4186 ± 16%     +29.2%       5409 ± 12%  interrupts.CPU2.RES:Rescheduling_interrupts
      5630 ±  7%     -14.9%       4788 ± 12%  interrupts.CPU3.RES:Rescheduling_interrupts
     98.92 ± 25%     -27.7%      71.56 ± 18%  sched_debug.cfs_rq:/.load_avg.stddev
    515190 ± 15%     +21.8%     627617 ±  8%  sched_debug.cpu.avg_idle.avg
     11670 ±  6%     +17.0%      13659        sched_debug.cpu.ttwu_count.min
      9696 ±  7%     +13.9%      11046 ±  4%  sched_debug.cpu.ttwu_local.min
    313013            +1.0%     316254        proc-vmstat.nr_active_anon
    309576            +1.1%     313054        proc-vmstat.nr_anon_pages
    313008            +1.0%     316246        proc-vmstat.nr_zone_active_anon
  22747924            -4.8%   21653355        proc-vmstat.numa_hit
  22747924            -4.8%   21653355        proc-vmstat.numa_local
  1.15e+09            -4.9%  1.093e+09        proc-vmstat.pgalloc_normal
  20511827            -4.9%   19515779        proc-vmstat.pgfault
  1.15e+09            -5.0%  1.093e+09        proc-vmstat.pgfree
   2206073            -4.9%    2097199        proc-vmstat.thp_fault_alloc
     82.50           -82.5        0.00        perf-profile.calltrace.cycles-pp.clear_subpage.clear_huge_page.do_huge_pmd_anonymous_page.__handle_mm_fault.handle_mm_fault
     80.56           -80.6        0.00        perf-profile.calltrace.cycles-pp.clear_page_erms.clear_subpage.clear_huge_page.do_huge_pmd_anonymous_page.__handle_mm_fault
      3.16            -0.3        2.87 ±  3%  perf-profile.calltrace.cycles-pp.alloc_pages_vma.do_huge_pmd_anonymous_page.__handle_mm_fault.handle_mm_fault.do_page_fault
      3.10            -0.3        2.81 ±  3%  perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.do_huge_pmd_anonymous_page.__handle_mm_fault
      3.12            -0.3        2.84 ±  3%  perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.alloc_pages_vma.do_huge_pmd_anonymous_page.__handle_mm_fault.handle_mm_fault
     93.63            +0.3       93.98        perf-profile.calltrace.cycles-pp.page_fault
     93.45            +0.4       93.81        perf-profile.calltrace.cycles-pp.do_page_fault.page_fault
     92.88            +0.4       93.31        perf-profile.calltrace.cycles-pp.handle_mm_fault.do_page_fault.page_fault
     92.65            +0.5       93.12        perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_page_fault.page_fault
     85.28            +0.6       85.83        perf-profile.calltrace.cycles-pp.clear_huge_page.do_huge_pmd_anonymous_page.__handle_mm_fault.handle_mm_fault.do_page_fault
     89.69            +0.7       90.36        perf-profile.calltrace.cycles-pp.do_huge_pmd_anonymous_page.__handle_mm_fault.handle_mm_fault.do_page_fault.page_fault
     82.68           -82.7        0.00        perf-profile.children.cycles-pp.clear_subpage
     82.08           -80.9        1.19        perf-profile.children.cycles-pp.clear_page_erms
      0.81            -0.6        0.20 ±  4%  perf-profile.children.cycles-pp.rcu_all_qs
      5.07            -0.4        4.64        perf-profile.children.cycles-pp.get_page_from_freelist
      5.20            -0.4        4.79        perf-profile.children.cycles-pp.__alloc_pages_nodemask
      4.92            -0.4        4.54 ±  2%  perf-profile.children.cycles-pp.alloc_pages_vma
      1.08            -0.1        0.96 ±  4%  perf-profile.children.cycles-pp._cond_resched
      0.40 ± 11%      -0.1        0.31 ±  4%  perf-profile.children.cycles-pp.mem_cgroup_try_charge_delay
      1.90            -0.1        1.81        perf-profile.children.cycles-pp.prep_new_page
      0.47 ±  4%      -0.1        0.41 ±  6%  perf-profile.children.cycles-pp.unmap_vmas
      0.46 ±  4%      -0.1        0.41 ±  6%  perf-profile.children.cycles-pp.unmap_page_range
      0.38 ±  8%      -0.0        0.33 ±  6%  perf-profile.children.cycles-pp.pte_alloc_one
      0.10 ±  7%      -0.0        0.07 ±  7%  perf-profile.children.cycles-pp.mem_cgroup_throttle_swaprate
      0.16 ±  5%      -0.0        0.13 ± 19%  perf-profile.children.cycles-pp._raw_spin_lock
      0.13 ±  6%      -0.0        0.10 ± 11%  perf-profile.children.cycles-pp.zap_huge_pmd
     93.72            +0.3       94.05        perf-profile.children.cycles-pp.page_fault
     93.53            +0.3       93.88        perf-profile.children.cycles-pp.do_page_fault
     92.94            +0.4       93.36        perf-profile.children.cycles-pp.handle_mm_fault
     92.72            +0.5       93.18        perf-profile.children.cycles-pp.__handle_mm_fault
     89.69            +0.7       90.36        perf-profile.children.cycles-pp.do_huge_pmd_anonymous_page
     85.41            +1.1       86.54        perf-profile.children.cycles-pp.clear_huge_page
     81.23           -80.1        1.17 ±  2%  perf-profile.self.cycles-pp.clear_page_erms
      0.60 ±  3%      -0.4        0.15 ±  2%  perf-profile.self.cycles-pp.rcu_all_qs
      1.56 ±  5%      -0.4        1.12 ±  4%  perf-profile.self.cycles-pp.___might_sleep
      2.44            -0.3        2.18 ±  4%  perf-profile.self.cycles-pp.get_page_from_freelist
      2.07 ±  4%      -0.1        1.96        perf-profile.self.cycles-pp.__free_pages_ok
      0.08 ±  6%      -0.0        0.05        perf-profile.self.cycles-pp.mem_cgroup_throttle_swaprate
      0.31 ±  8%      +0.1        0.46 ±  2%  perf-profile.self.cycles-pp._cond_resched
      0.21 ±  8%     +83.9       84.10        perf-profile.self.cycles-pp.clear_huge_page



***************************************************************************************************
lkp-ivb-d02: 4 threads Intel(R) Core(TM) i3-3220 CPU @ 3.30GHz with 8G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/rootfs/runtime/size/tbox_group/test/testcase/ucode:
  gcc-7/performance/x86_64-rhel-7.6/debian-x86_64-20191114.cgz/300s/8T/lkp-ivb-d02/anon-w-seq-mt/vm-scalability/0x21

commit: 
  8632e9b564 (" hyperv-fixes for 5.7-rc1")
  9266baf8ee ("mm: Optimized hugepage zeroing & copying from user")

8632e9b5645bbc23 9266baf8eea588a78e5a0aa4ccd 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
      0.00            +1.8%       0.00        vm-scalability.free_time
   1148096            +8.4%    1244718        vm-scalability.median
   4588586            +7.8%    4948645        vm-scalability.throughput
   1475456            +7.5%    1586296        vm-scalability.time.minor_page_faults
    270.13           -15.6%     227.88 ±  2%  vm-scalability.time.system_time
    868.34            +4.7%     908.82        vm-scalability.time.user_time
 1.326e+09            +7.8%   1.43e+09        vm-scalability.workload
      0.01 ± 37%      +0.0        0.02 ± 76%  mpstat.cpu.all.soft%
     23.34            -3.5       19.88 ±  2%  mpstat.cpu.all.sys%
     73.00            +5.1%      76.75        vmstat.cpu.us
      6969            +5.2%       7329 ±  2%  vmstat.system.cs
    627528 ±  2%     -16.5%     524174 ±  4%  sched_debug.cpu.avg_idle.avg
    -12.75           +25.5%     -16.00        sched_debug.cpu.nr_uninterruptible.min
     10.19 ±  6%     +24.2%      12.65 ± 18%  sched_debug.cpu.nr_uninterruptible.stddev
  19223068 ± 15%     -21.8%   15037169 ± 10%  cpuidle.C6.time
     33333 ±  3%     -16.3%      27901 ±  6%  cpuidle.C6.usage
      8465 ± 39%     -58.8%       3487 ± 15%  cpuidle.POLL.time
      2649 ± 23%     -54.6%       1203 ± 21%  cpuidle.POLL.usage
     26729 ±  7%     +78.6%      47740 ± 39%  softirqs.CPU0.RCU
      4574 ± 15%     +79.7%       8219 ± 55%  softirqs.CPU1.NET_RX
     28386 ± 10%     +63.3%      46347 ± 40%  softirqs.CPU1.RCU
     27498 ±  6%     +75.2%      48178 ± 40%  softirqs.CPU2.RCU
     27869 ±  5%     +73.8%      48433 ± 39%  softirqs.CPU3.RCU
      4577 ± 15%     +79.6%       8222 ± 55%  softirqs.NET_RX
    110484 ±  4%     +72.6%     190699 ± 39%  softirqs.RCU
   3455928 ±  2%     -36.0%    2212394        meminfo.Active
   3452870 ±  2%     -36.0%    2209318        meminfo.Active(anon)
   3281881 ±  2%     -37.5%    2049676        meminfo.AnonHugePages
   3425083 ±  2%     -35.9%    2195653        meminfo.AnonPages
     52607 ± 16%    +165.2%     139515 ±  2%  meminfo.CmaFree
   3259754 ±  2%     +38.2%    4504331        meminfo.MemAvailable
   3368788 ±  2%     +36.9%    4613377        meminfo.MemFree
   4662470 ±  2%     -26.7%    3417882        meminfo.Memused
     10268           -24.1%       7794        meminfo.PageTables
      8448 ± 16%     +82.5%      15416 ± 55%  interrupts.27:PCI-MSI.1572864-edge.eth0
      4211           -13.1%       3661        interrupts.CAL:Function_call_interrupts
      1058           -14.5%     904.50 ±  2%  interrupts.CPU0.CAL:Function_call_interrupts
      8448 ± 16%     +82.5%      15416 ± 55%  interrupts.CPU1.27:PCI-MSI.1572864-edge.eth0
      1042 ±  4%     -14.6%     889.75 ±  3%  interrupts.CPU1.CAL:Function_call_interrupts
      6167           +52.0%       9373 ± 29%  interrupts.CPU1.NMI:Non-maskable_interrupts
      6167           +52.0%       9373 ± 29%  interrupts.CPU1.PMI:Performance_monitoring_interrupts
      1076           -10.1%     968.25 ±  2%  interrupts.CPU2.CAL:Function_call_interrupts
      1034 ±  2%     -13.1%     898.75 ±  4%  interrupts.CPU3.CAL:Function_call_interrupts
      6172           +53.3%       9464 ± 32%  interrupts.CPU3.NMI:Non-maskable_interrupts
      6172           +53.3%       9464 ± 32%  interrupts.CPU3.PMI:Performance_monitoring_interrupts
    825701 ±  3%     -33.6%     548074        proc-vmstat.nr_active_anon
    810670 ±  3%     -32.9%     543797        proc-vmstat.nr_anon_pages
      1516 ±  3%     -34.5%     992.25        proc-vmstat.nr_anon_transparent_hugepages
     85789 ±  3%     +30.6%     112073        proc-vmstat.nr_dirty_background_threshold
    171789 ±  3%     +30.6%     224421        proc-vmstat.nr_dirty_threshold
     15623 ± 19%    +127.4%      35525        proc-vmstat.nr_free_cma
    879784 ±  3%     +31.6%    1157608        proc-vmstat.nr_free_pages
      3558            -0.7%       3535        proc-vmstat.nr_kernel_stack
      2481 ±  2%     -21.7%       1944        proc-vmstat.nr_page_table_pages
    825696 ±  3%     -33.6%     548054        proc-vmstat.nr_zone_active_anon
   2326268            +6.5%    2476331        proc-vmstat.numa_hit
   2326268            +6.5%    2476331        proc-vmstat.numa_local
  72116983            +5.5%   76117229 ±  3%  proc-vmstat.pgalloc_dma32
 2.237e+08            +8.6%  2.429e+08        proc-vmstat.pgalloc_normal
   1859214            +5.9%    1969537        proc-vmstat.pgfault
 2.948e+08            +8.1%  3.186e+08        proc-vmstat.pgfree
    574297            +7.8%     619284        proc-vmstat.thp_fault_alloc
     21.56 ±  7%     -21.6        0.00        perf-profile.calltrace.cycles-pp.clear_subpage.clear_huge_page.do_huge_pmd_anonymous_page.__handle_mm_fault.handle_mm_fault
     20.80 ±  7%     -20.8        0.00        perf-profile.calltrace.cycles-pp.clear_page_erms.clear_subpage.clear_huge_page.do_huge_pmd_anonymous_page.__handle_mm_fault
     23.47 ±  7%      -5.2       18.26 ±  2%  perf-profile.calltrace.cycles-pp.do_huge_pmd_anonymous_page.__handle_mm_fault.handle_mm_fault.do_page_fault.page_fault
     22.35 ±  7%      -5.2       17.18 ±  2%  perf-profile.calltrace.cycles-pp.clear_huge_page.do_huge_pmd_anonymous_page.__handle_mm_fault.handle_mm_fault.do_page_fault
     22.65 ±  3%      -4.5       18.18        perf-profile.calltrace.cycles-pp.handle_mm_fault.do_page_fault.page_fault.do_access
     22.74 ±  3%      -4.5       18.27        perf-profile.calltrace.cycles-pp.do_page_fault.page_fault.do_access
     22.61 ±  3%      -4.5       18.14        perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_page_fault.page_fault.do_access
     22.77 ±  3%      -4.5       18.31        perf-profile.calltrace.cycles-pp.page_fault.do_access
      0.73 ±  8%      -0.1        0.63 ±  5%  perf-profile.calltrace.cycles-pp.alloc_pages_vma.do_huge_pmd_anonymous_page.__handle_mm_fault.handle_mm_fault.do_page_fault
      0.72 ±  8%      -0.1        0.62 ±  5%  perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.alloc_pages_vma.do_huge_pmd_anonymous_page.__handle_mm_fault.handle_mm_fault
      0.70 ±  9%      -0.1        0.61 ±  5%  perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.do_huge_pmd_anonymous_page.__handle_mm_fault
     62.06 ±  2%      +2.1       64.16        perf-profile.calltrace.cycles-pp.do_rw_once
     21.78 ±  7%     -21.8        0.00        perf-profile.children.cycles-pp.clear_subpage
     21.35 ±  7%     -21.3        0.06 ± 16%  perf-profile.children.cycles-pp.clear_page_erms
     23.62 ±  7%      -5.2       18.45 ±  2%  perf-profile.children.cycles-pp.do_huge_pmd_anonymous_page
     23.93 ±  6%      -5.1       18.82 ±  2%  perf-profile.children.cycles-pp.do_page_fault
     23.82 ±  6%      -5.1       18.73 ±  2%  perf-profile.children.cycles-pp.handle_mm_fault
     23.98 ±  6%      -5.1       18.89 ±  2%  perf-profile.children.cycles-pp.page_fault
     23.77 ±  7%      -5.1       18.69 ±  2%  perf-profile.children.cycles-pp.__handle_mm_fault
     22.54 ±  7%      -5.0       17.52        perf-profile.children.cycles-pp.clear_huge_page
     48.63            -1.0       47.68        perf-profile.children.cycles-pp.do_access
      1.40 ±  8%      -0.4        1.03 ± 19%  perf-profile.children.cycles-pp.apic_timer_interrupt
      1.30 ±  7%      -0.4        0.95 ± 21%  perf-profile.children.cycles-pp.smp_apic_timer_interrupt
      1.09 ±  9%      -0.3        0.75 ± 22%  perf-profile.children.cycles-pp.hrtimer_interrupt
      0.91 ± 11%      -0.3        0.64 ± 22%  perf-profile.children.cycles-pp.__hrtimer_run_queues
      0.67 ±  9%      -0.2        0.46 ± 20%  perf-profile.children.cycles-pp.tick_sched_timer
      0.58 ± 10%      -0.2        0.39 ± 18%  perf-profile.children.cycles-pp.tick_sched_handle
      0.56 ± 10%      -0.2        0.37 ± 20%  perf-profile.children.cycles-pp.update_process_times
      0.21 ±  6%      -0.1        0.07 ± 17%  perf-profile.children.cycles-pp.rcu_all_qs
      0.34 ±  6%      -0.1        0.23 ± 12%  perf-profile.children.cycles-pp.scheduler_tick
      0.46 ± 13%      -0.1        0.35 ±  8%  perf-profile.children.cycles-pp.___might_sleep
      0.80 ±  7%      -0.1        0.71 ±  6%  perf-profile.children.cycles-pp.alloc_pages_vma
      0.25 ±  8%      -0.1        0.17 ± 14%  perf-profile.children.cycles-pp.task_tick_fair
      0.28 ±  9%      -0.1        0.21 ±  5%  perf-profile.children.cycles-pp._cond_resched
      0.15 ± 11%      -0.0        0.11 ± 22%  perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode
      0.08 ±  8%      +0.0        0.12 ± 10%  perf-profile.children.cycles-pp.__do_execve_file
      0.08 ±  8%      +0.0        0.12 ± 12%  perf-profile.children.cycles-pp.__x64_sys_execve
      0.08 ± 10%      +0.0        0.12 ±  8%  perf-profile.children.cycles-pp.execve
      0.03 ±102%      +0.0        0.08 ± 19%  perf-profile.children.cycles-pp.search_binary_handler
      0.01 ±173%      +0.1        0.06 ±  6%  perf-profile.children.cycles-pp.drm_fb_helper_dirty
      0.11 ± 33%      +0.1        0.18 ± 16%  perf-profile.children.cycles-pp.handle_pte_fault
      1.27 ±  7%      +0.2        1.47 ±  4%  perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
      1.27 ±  7%      +0.2        1.47 ±  4%  perf-profile.children.cycles-pp.do_syscall_64
     63.21            +3.4       66.66        perf-profile.children.cycles-pp.do_rw_once
     20.97 ±  6%     -20.9        0.06 ± 16%  perf-profile.self.cycles-pp.clear_page_erms
      0.44 ± 16%      -0.2        0.28 ± 11%  perf-profile.self.cycles-pp.___might_sleep
      0.16 ± 13%      -0.1        0.05 ± 58%  perf-profile.self.cycles-pp.rcu_all_qs
      0.08 ± 23%      -0.1        0.03 ±100%  perf-profile.self.cycles-pp.task_tick_fair
      0.06 ± 14%      +0.0        0.08 ± 10%  perf-profile.self.cycles-pp._raw_spin_lock_irqsave
     17.35 ±  5%      +1.8       19.11        perf-profile.self.cycles-pp.do_access
     55.41            +3.1       58.55        perf-profile.self.cycles-pp.do_rw_once
      0.09 ± 27%     +16.8       16.94 ±  2%  perf-profile.self.cycles-pp.clear_huge_page



***************************************************************************************************
lkp-csl-2sp6: 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 256G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/rootfs/runtime/size/tbox_group/test/testcase/ucode:
  gcc-7/performance/x86_64-rhel-7.6/debian-x86_64-20191114.cgz/300s/8T/lkp-csl-2sp6/anon-cow-seq-hugetlb/vm-scalability/0x500002c

commit: 
  8632e9b564 (" hyperv-fixes for 5.7-rc1")
  9266baf8ee ("mm: Optimized hugepage zeroing & copying from user")

8632e9b5645bbc23 9266baf8eea588a78e5a0aa4ccd 
---------------- --------------------------- 
       fail:runs  %reproduction    fail:runs
           |             |             |    
           :4           75%           3:4     dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x
           :4           75%           3:4     dmesg.WARNING:stack_recursion
         %stddev     %change         %stddev
             \          |                \  
    361704            -2.6%     352245        vm-scalability.median
  36376079            -4.6%   34694751        vm-scalability.throughput
   4265227            -5.4%    4036714        vm-scalability.time.minor_page_faults
     13472            -2.7%      13115        vm-scalability.time.system_time
      8845            +2.1%       9035        vm-scalability.time.user_time
    354016            -5.3%     335133        vm-scalability.time.voluntary_context_switches
 9.105e+09            -5.4%  8.616e+09        vm-scalability.workload
      4848            -2.0%       4751        vmstat.system.cs
   2477460 ±  3%      -8.2%    2274167 ±  4%  numa-numastat.node0.local_node
   2508460 ±  3%      -8.3%    2301032 ±  4%  numa-numastat.node0.numa_hit
      1386 ±  4%      -7.1%       1288 ±  5%  slabinfo.Acpi-Parse.active_objs
      1386 ±  4%      -7.1%       1288 ±  5%  slabinfo.Acpi-Parse.num_objs
      3371 ± 11%     +26.7%       4271 ± 10%  slabinfo.eventpoll_pwq.active_objs
      3371 ± 11%     +26.7%       4271 ± 10%  slabinfo.eventpoll_pwq.num_objs
    812.33 ±  9%     -15.8%     684.33 ±  4%  slabinfo.kmem_cache_node.active_objs
    852.33 ±  9%     -14.9%     725.00 ±  4%  slabinfo.kmem_cache_node.num_objs
   3993284            -5.4%    3778602        proc-vmstat.htlb_buddy_alloc_success
    179.00            -3.5%     172.67        proc-vmstat.nr_dirtied
     45924            -1.1%      45436        proc-vmstat.nr_shmem
    166.00            -3.6%     160.00        proc-vmstat.nr_written
   5075790            -4.8%    4832494        proc-vmstat.numa_hit
   5044600            -4.8%    4801279        proc-vmstat.numa_local
 2.046e+09            -5.4%  1.936e+09        proc-vmstat.pgalloc_normal
   5095148            -4.6%    4860017        proc-vmstat.pgfault
 2.045e+09            -5.3%  1.936e+09        proc-vmstat.pgfree
      1299 ± 71%    +185.8%       3715 ± 35%  sched_debug.cfs_rq:/.load.min
      0.12 ± 71%    +181.8%       0.34 ± 35%  sched_debug.cfs_rq:/.nr_running.min
      0.29 ±  5%     -33.3%       0.19 ± 11%  sched_debug.cfs_rq:/.nr_running.stddev
      1164 ±  6%     +20.0%       1397 ±  4%  sched_debug.cfs_rq:/.util_avg.max
    244.31 ±  4%     -12.5%     213.67 ±  2%  sched_debug.cfs_rq:/.util_avg.stddev
      0.12 ± 71%    +181.8%       0.34 ± 35%  sched_debug.cfs_rq:/.util_est_enqueued.min
      8.71 ± 35%     +62.1%      14.13 ± 20%  sched_debug.cpu.clock.stddev
      8.71 ± 35%     +62.1%      14.13 ± 20%  sched_debug.cpu.clock_task.stddev
      2263 ± 13%     -33.6%       1503 ± 12%  sched_debug.cpu.curr->pid.stddev
      1.18           +30.2%       1.53 ±  9%  sched_debug.cpu.nr_running.max
      0.12 ± 71%    +236.4%       0.41 ± 16%  sched_debug.cpu.nr_running.min
      0.30 ±  2%     -24.7%       0.23 ± 12%  sched_debug.cpu.nr_running.stddev
      0.06 ± 18%     -75.4%       0.02 ± 40%  sched_debug.cpu.nr_uninterruptible.avg
    -50.38           -19.7%     -40.47        sched_debug.cpu.nr_uninterruptible.min
      9247 ± 13%     +20.9%      11175 ±  7%  sched_debug.cpu.ttwu_count.max
     13116 ±  9%     +12.9%      14811 ±  6%  softirqs.CPU34.SCHED
     13634 ±  9%     -14.8%      11609 ± 10%  softirqs.CPU4.SCHED
    123480 ±  6%     -10.8%     110147        softirqs.CPU4.TIMER
    123091 ±  6%     -10.4%     110308        softirqs.CPU52.TIMER
     12976 ± 13%     -13.9%      11174 ± 14%  softirqs.CPU53.SCHED
     12600 ± 10%     -12.2%      11058 ± 15%  softirqs.CPU56.SCHED
     13147 ± 13%     -14.2%      11275 ± 13%  softirqs.CPU57.SCHED
     13284 ± 14%     -18.0%      10898 ± 15%  softirqs.CPU58.SCHED
    121769 ±  6%     -10.3%     109258        softirqs.CPU58.TIMER
     13140 ± 12%     -19.6%      10561 ± 15%  softirqs.CPU60.SCHED
     13017 ± 13%     -13.4%      11274 ± 12%  softirqs.CPU64.SCHED
     12951 ± 12%     -17.2%      10724 ± 14%  softirqs.CPU68.SCHED
     13327 ± 14%     -15.8%      11216 ± 15%  softirqs.CPU69.SCHED
     12891 ± 14%     +14.7%      14789 ±  7%  softirqs.CPU84.SCHED
     13383 ±  9%     -10.6%      11968 ±  9%  softirqs.CPU9.SCHED
     39.20           -39.2        0.00        perf-profile.calltrace.cycles-pp.copy_subpage.copy_user_huge_page.hugetlb_cow.hugetlb_fault.handle_mm_fault
     38.70           -38.7        0.00        perf-profile.calltrace.cycles-pp.copy_page.copy_subpage.copy_user_huge_page.hugetlb_cow.hugetlb_fault
     83.46           -17.1       66.37 ±  9%  perf-profile.calltrace.cycles-pp.do_access
     52.26           -12.6       39.62 ±  9%  perf-profile.calltrace.cycles-pp.page_fault.do_access
     52.22           -12.6       39.59 ±  9%  perf-profile.calltrace.cycles-pp.do_page_fault.page_fault.do_access
     52.16           -12.6       39.54 ±  9%  perf-profile.calltrace.cycles-pp.handle_mm_fault.do_page_fault.page_fault.do_access
     52.13           -12.6       39.52 ±  9%  perf-profile.calltrace.cycles-pp.hugetlb_fault.handle_mm_fault.do_page_fault.page_fault.do_access
     14.42 ±  2%      -3.6       10.85 ±  7%  perf-profile.calltrace.cycles-pp.do_rw_once
      0.69 ±  7%      -0.3        0.37 ± 70%  perf-profile.calltrace.cycles-pp.alloc_huge_page.hugetlb_cow.hugetlb_fault.handle_mm_fault.do_page_fault
      0.00            +0.8        0.80 ± 24%  perf-profile.calltrace.cycles-pp.apic_timer_interrupt.copy_page.copy_user_huge_page.hugetlb_cow.hugetlb_fault
     39.37            +6.0       45.35        perf-profile.calltrace.cycles-pp.copy_user_huge_page.hugetlb_cow.hugetlb_fault.handle_mm_fault.do_page_fault
     40.32            +6.1       46.44        perf-profile.calltrace.cycles-pp.hugetlb_cow.hugetlb_fault.handle_mm_fault.do_page_fault.page_fault
      6.40 ± 36%     +18.0       24.41 ± 23%  perf-profile.calltrace.cycles-pp.hugetlb_fault.handle_mm_fault.do_page_fault.page_fault
      6.41 ± 36%     +18.0       24.44 ± 23%  perf-profile.calltrace.cycles-pp.handle_mm_fault.do_page_fault.page_fault
      6.42 ± 36%     +18.0       24.47 ± 23%  perf-profile.calltrace.cycles-pp.do_page_fault.page_fault
      6.42 ± 36%     +18.1       24.48 ± 23%  perf-profile.calltrace.cycles-pp.page_fault
      0.00           +44.6       44.64        perf-profile.calltrace.cycles-pp.copy_page.copy_user_huge_page.hugetlb_cow.hugetlb_fault.handle_mm_fault
     39.21           -39.2        0.00        perf-profile.children.cycles-pp.copy_subpage
     72.47           -16.0       56.47 ±  9%  perf-profile.children.cycles-pp.do_access
     25.92 ±  2%      -4.8       21.11 ±  8%  perf-profile.children.cycles-pp.do_rw_once
      0.11            +0.0        0.13 ±  6%  perf-profile.children.cycles-pp.native_irq_return_iret
      0.07 ± 11%      +0.0        0.10 ±  4%  perf-profile.children.cycles-pp.native_write_msr
      0.11 ±  4%      +0.0        0.14 ±  3%  perf-profile.children.cycles-pp.update_and_free_page
      0.28 ±  5%      +0.0        0.32        perf-profile.children.cycles-pp.process_one_work
      0.27 ±  4%      +0.0        0.31        perf-profile.children.cycles-pp.drm_fb_helper_dirty_work
      0.37 ±  3%      +0.0        0.41 ±  6%  perf-profile.children.cycles-pp.task_tick_fair
      0.29 ±  4%      +0.0        0.33        perf-profile.children.cycles-pp.ret_from_fork
      0.29 ±  4%      +0.0        0.33        perf-profile.children.cycles-pp.kthread
      0.28 ±  5%      +0.0        0.32        perf-profile.children.cycles-pp.worker_thread
      0.27 ±  4%      +0.0        0.32        perf-profile.children.cycles-pp.memcpy_erms
      0.09 ± 24%      +0.0        0.13 ± 14%  perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
      0.02 ±141%      +0.1        0.07 ±  7%  perf-profile.children.cycles-pp.__intel_pmu_enable_all
      0.00            +0.1        0.05        perf-profile.children.cycles-pp.run_local_timers
      0.02 ±141%      +0.1        0.07 ± 23%  perf-profile.children.cycles-pp.menu_select
      0.58 ±  2%      +0.1        0.66 ±  2%  perf-profile.children.cycles-pp.__alloc_pages_nodemask
      0.56 ±  2%      +0.1        0.64        perf-profile.children.cycles-pp.get_page_from_freelist
      0.62 ±  3%      +0.1        0.72        perf-profile.children.cycles-pp.alloc_fresh_huge_page
      0.05 ± 74%      +0.1        0.14 ± 32%  perf-profile.children.cycles-pp.start_kernel
      1.02 ±  2%      +0.1        1.13 ±  6%  perf-profile.children.cycles-pp.__hrtimer_run_queues
      0.71            +0.1        0.82 ±  3%  perf-profile.children.cycles-pp.alloc_surplus_huge_page
      0.84            +0.1        0.96 ±  3%  perf-profile.children.cycles-pp.alloc_huge_page
      1.26 ±  2%      +0.1        1.40 ±  2%  perf-profile.children.cycles-pp.hrtimer_interrupt
      1.47 ±  2%      +0.1        1.62        perf-profile.children.cycles-pp.smp_apic_timer_interrupt
      1.91            +0.2        2.14        perf-profile.children.cycles-pp.apic_timer_interrupt
     58.58 ±  3%      +5.4       63.99 ±  4%  perf-profile.children.cycles-pp.handle_mm_fault
     58.60 ±  3%      +5.4       64.02 ±  4%  perf-profile.children.cycles-pp.hugetlb_fault
     58.70 ±  3%      +5.4       64.12 ±  4%  perf-profile.children.cycles-pp.page_fault
     58.65 ±  3%      +5.4       64.08 ±  4%  perf-profile.children.cycles-pp.do_page_fault
     39.38            +6.0       45.36        perf-profile.children.cycles-pp.copy_user_huge_page
     39.05            +6.0       45.07        perf-profile.children.cycles-pp.copy_page
     40.33            +6.1       46.44        perf-profile.children.cycles-pp.hugetlb_cow
     25.22 ±  2%      -4.6       20.61 ±  8%  perf-profile.self.cycles-pp.do_rw_once
      8.36            -2.1        6.29 ±  8%  perf-profile.self.cycles-pp.do_access
      0.11            +0.0        0.13 ±  6%  perf-profile.self.cycles-pp.native_irq_return_iret
      0.07 ± 11%      +0.0        0.10 ±  4%  perf-profile.self.cycles-pp.native_write_msr
      0.11 ±  4%      +0.0        0.14 ±  3%  perf-profile.self.cycles-pp.update_and_free_page
      0.27 ±  5%      +0.0        0.31        perf-profile.self.cycles-pp.memcpy_erms
      0.00            +0.1        0.05        perf-profile.self.cycles-pp.run_local_timers
      0.33            +0.1        0.40        perf-profile.self.cycles-pp.get_page_from_freelist
      0.00            +0.1        0.08 ± 10%  perf-profile.self.cycles-pp.clear_huge_page
      0.00            +0.1        0.12        perf-profile.self.cycles-pp.copy_user_huge_page
     38.43            +5.9       44.38        perf-profile.self.cycles-pp.copy_page
    291.00 ±  9%     -11.9%     256.33 ±  3%  interrupts.35:PCI-MSI.31981568-edge.i40e-0000:3d:00.0:misc
    297.00 ±116%     -99.0%       3.00 ±141%  interrupts.46:PCI-MSI.31981579-edge.i40e-eth0-TxRx-10
    296.67 ±116%     -99.1%       2.67 ±141%  interrupts.CPU10.46:PCI-MSI.31981579-edge.i40e-eth0-TxRx-10
    501.00 ±  3%     +22.0%     611.00 ±  8%  interrupts.CPU12.RES:Rescheduling_interrupts
      6041 ±  7%     -37.7%       3766 ± 20%  interrupts.CPU2.NMI:Non-maskable_interrupts
      6041 ±  7%     -37.7%       3766 ± 20%  interrupts.CPU2.PMI:Performance_monitoring_interrupts
      5468 ±  8%     +10.8%       6057 ± 10%  interrupts.CPU21.NMI:Non-maskable_interrupts
      5468 ±  8%     +10.8%       6057 ± 10%  interrupts.CPU21.PMI:Performance_monitoring_interrupts
      1547 ± 63%    +186.7%       4435 ± 17%  interrupts.CPU24.TLB:TLB_shootdowns
    585.67           -23.1%     450.33 ± 20%  interrupts.CPU25.CAL:Function_call_interrupts
      6241 ±  4%     -23.6%       4767 ± 26%  interrupts.CPU25.NMI:Non-maskable_interrupts
      6241 ±  4%     -23.6%       4767 ± 26%  interrupts.CPU25.PMI:Performance_monitoring_interrupts
      1089 ± 68%     -56.9%     469.67 ±  2%  interrupts.CPU26.RES:Rescheduling_interrupts
      6424           -11.8%       5667 ±  4%  interrupts.CPU28.NMI:Non-maskable_interrupts
      6424           -11.8%       5667 ±  4%  interrupts.CPU28.PMI:Performance_monitoring_interrupts
      6076 ±  8%     -22.9%       4684 ± 20%  interrupts.CPU30.NMI:Non-maskable_interrupts
      6076 ±  8%     -22.9%       4684 ± 20%  interrupts.CPU30.PMI:Performance_monitoring_interrupts
    895.33 ± 73%    +105.6%       1840 ±  3%  interrupts.CPU30.TLB:TLB_shootdowns
      6224 ±  9%     -23.3%       4777 ± 26%  interrupts.CPU31.NMI:Non-maskable_interrupts
      6224 ±  9%     -23.3%       4777 ± 26%  interrupts.CPU31.PMI:Performance_monitoring_interrupts
    529.00 ±  6%     -27.2%     385.00 ± 26%  interrupts.CPU38.RES:Rescheduling_interrupts
    970.00 ± 69%    +103.5%       1974 ±  9%  interrupts.CPU38.TLB:TLB_shootdowns
      4676 ± 26%     +33.3%       6233 ± 10%  interrupts.CPU4.NMI:Non-maskable_interrupts
      4676 ± 26%     +33.3%       6233 ± 10%  interrupts.CPU4.PMI:Performance_monitoring_interrupts
    518.67 ± 22%     -25.4%     387.00 ± 25%  interrupts.CPU40.RES:Rescheduling_interrupts
    450.67 ± 20%     -23.0%     347.00 ± 21%  interrupts.CPU41.RES:Rescheduling_interrupts
      6379 ±  7%      -8.6%       5832 ±  7%  interrupts.CPU42.NMI:Non-maskable_interrupts
      6379 ±  7%      -8.6%       5832 ±  7%  interrupts.CPU42.PMI:Performance_monitoring_interrupts
    509.67 ± 13%     -34.7%     332.67 ± 22%  interrupts.CPU42.RES:Rescheduling_interrupts
    527.33 ±  9%     -44.6%     292.00 ± 19%  interrupts.CPU43.RES:Rescheduling_interrupts
    545.33 ± 27%     -41.1%     321.33 ± 10%  interrupts.CPU45.RES:Rescheduling_interrupts
    507.67 ± 25%     -34.6%     332.00 ± 22%  interrupts.CPU46.RES:Rescheduling_interrupts
    435.00 ± 22%     -21.5%     341.67 ±  6%  interrupts.CPU48.RES:Rescheduling_interrupts
      1142 ± 65%     -72.2%     317.33 ± 26%  interrupts.CPU56.RES:Rescheduling_interrupts
    602.00 ± 64%     -56.0%     265.00 ±  5%  interrupts.CPU60.RES:Rescheduling_interrupts
    282.00 ±  6%     -16.2%     236.33 ±  2%  interrupts.CPU61.RES:Rescheduling_interrupts
     64.33 ± 67%    +293.8%     253.33 ± 43%  interrupts.CPU61.TLB:TLB_shootdowns
    771.00 ± 62%     -66.1%     261.33 ± 19%  interrupts.CPU66.RES:Rescheduling_interrupts
     81.00 ± 83%    +135.0%     190.33 ± 27%  interrupts.CPU66.TLB:TLB_shootdowns
      4613 ± 21%     +35.3%       6242 ± 10%  interrupts.CPU68.NMI:Non-maskable_interrupts
      4613 ± 21%     +35.3%       6242 ± 10%  interrupts.CPU68.PMI:Performance_monitoring_interrupts
      4684 ± 35%     +35.8%       6359 ±  8%  interrupts.CPU69.NMI:Non-maskable_interrupts
      4684 ± 35%     +35.8%       6359 ±  8%  interrupts.CPU69.PMI:Performance_monitoring_interrupts
    291.00 ±  9%     -11.9%     256.33 ±  3%  interrupts.CPU7.35:PCI-MSI.31981568-edge.i40e-0000:3d:00.0:misc
    102.00 ± 71%    +105.2%     209.33 ± 26%  interrupts.CPU70.TLB:TLB_shootdowns
    245.00 ± 61%    +156.5%     628.33 ±  5%  interrupts.CPU72.TLB:TLB_shootdowns
    140.67 ± 53%    +183.9%     399.33 ± 47%  interrupts.CPU74.TLB:TLB_shootdowns
      6394 ±  2%      -8.2%       5870 ±  6%  interrupts.CPU76.NMI:Non-maskable_interrupts
      6394 ±  2%      -8.2%       5870 ±  6%  interrupts.CPU76.PMI:Performance_monitoring_interrupts
    173.33 ± 54%    +251.9%     610.00 ± 32%  interrupts.CPU76.TLB:TLB_shootdowns
     74.00 ± 36%    +544.6%     477.00 ± 14%  interrupts.CPU77.TLB:TLB_shootdowns
    317.00 ± 24%     -26.4%     233.33 ± 20%  interrupts.CPU79.RES:Rescheduling_interrupts
    114.67 ±103%    +126.7%     260.00 ± 23%  interrupts.CPU79.TLB:TLB_shootdowns
    152.00 ± 85%    +110.3%     319.67 ± 11%  interrupts.CPU82.TLB:TLB_shootdowns
    284.67           -31.6%     194.67 ± 32%  interrupts.CPU83.RES:Rescheduling_interrupts
    267.67 ±  8%     -19.7%     215.00 ± 17%  interrupts.CPU84.RES:Rescheduling_interrupts
     99.67 ±115%    +134.1%     233.33 ± 40%  interrupts.CPU84.TLB:TLB_shootdowns
     76.00 ± 87%    +200.4%     228.33 ± 32%  interrupts.CPU86.TLB:TLB_shootdowns
      6324 ±  6%     -21.9%       4936 ± 29%  interrupts.CPU87.NMI:Non-maskable_interrupts
      6324 ±  6%     -21.9%       4936 ± 29%  interrupts.CPU87.PMI:Performance_monitoring_interrupts
     38.67 ±112%    +550.0%     251.33 ± 58%  interrupts.CPU87.TLB:TLB_shootdowns
    720.33 ± 66%     -69.8%     217.67 ± 19%  interrupts.CPU89.RES:Rescheduling_interrupts
    309.67 ± 14%     -35.5%     199.67 ± 19%  interrupts.CPU90.RES:Rescheduling_interrupts
     31.67 ± 70%    +533.7%     200.67 ± 15%  interrupts.CPU92.TLB:TLB_shootdowns
      6189 ±  9%     -24.8%       4656 ± 29%  interrupts.CPU94.NMI:Non-maskable_interrupts
      6189 ±  9%     -24.8%       4656 ± 29%  interrupts.CPU94.PMI:Performance_monitoring_interrupts
     73.00 ± 65%    +444.3%     397.33 ± 24%  interrupts.CPU94.TLB:TLB_shootdowns
      6313 ±  5%      -9.7%       5703 ±  4%  interrupts.CPU95.NMI:Non-maskable_interrupts
      6313 ±  5%      -9.7%       5703 ±  4%  interrupts.CPU95.PMI:Performance_monitoring_interrupts
     49044 ± 10%     -15.0%      41667 ± 12%  interrupts.RES:Rescheduling_interrupts



***************************************************************************************************
lkp-ivb-d01: 8 threads Intel(R) Core(TM) i7-3770K CPU @ 3.50GHz with 16G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode:
  gcc-7/performance/x86_64-rhel-7.6/thread/100%/debian-x86_64-20191114.cgz/lkp-ivb-d01/page_fault1/will-it-scale/0x21

commit: 
  8632e9b564 (" hyperv-fixes for 5.7-rc1")
  9266baf8ee ("mm: Optimized hugepage zeroing & copying from user")

8632e9b5645bbc23 9266baf8eea588a78e5a0aa4ccd 
---------------- --------------------------- 
       fail:runs  %reproduction    fail:runs
           |             |             |    
           :4           25%           1:4     dmesg.RIP:cpuidle_enter_state
           :4           25%           1:4     dmesg.RIP:simple_getattr
           :4           25%           1:4     kmsg.b135d8f>]usb_hcd_irq
          1:4          -25%            :4     kmsg.c48>]usb_hcd_irq
          1:4          -25%            :4     kmsg.ca#>]usb_hcd_irq
          1:4          -25%            :4     kmsg.cb32>]usb_hcd_irq
           :4           25%           1:4     kmsg.d428ae1>]usb_hcd_irq
          1:4          -25%            :4     kmsg.ddef>]usb_hcd_irq
         %stddev     %change         %stddev
             \          |                \  
    534714            -3.5%     515893        will-it-scale.per_thread_ops
   2517464            -3.5%    2428934        will-it-scale.time.minor_page_faults
    518727            -4.8%     493620        will-it-scale.time.voluntary_context_switches
   4277722            -3.5%    4127153        will-it-scale.workload
      9481            -1.3%       9358        vmstat.system.cs
   2764362          +196.4%    8193435 ± 28%  cpuidle.C1.time
     63195 ±  3%    +126.3%     143040 ± 41%  cpuidle.C1.usage
     18199 ±  7%     +12.8%      20532 ±  9%  softirqs.CPU3.RCU
     17571 ± 10%     +21.0%      21255 ±  7%  softirqs.CPU5.RCU
    248287            -6.0%     233281        interrupts.CAL:Function_call_interrupts
      2273 ± 17%     -18.5%       1852        interrupts.CPU1.RES:Rescheduling_interrupts
      1965 ±  7%      -7.7%       1815 ±  3%  interrupts.CPU2.RES:Rescheduling_interrupts
      4.43 ±  7%     -19.6%       3.56 ± 15%  sched_debug.cfs_rq:/.nr_spread_over.avg
     32.42 ± 12%     -41.5%      18.96 ± 32%  sched_debug.cfs_rq:/.nr_spread_over.max
     10.63 ± 12%     -39.0%       6.48 ± 25%  sched_debug.cfs_rq:/.nr_spread_over.stddev
   5397479            -3.2%    5222071        proc-vmstat.numa_hit
   5397479            -3.2%    5222071        proc-vmstat.numa_local
 1.289e+09            -3.5%  1.243e+09        proc-vmstat.pgalloc_normal
   2902685            -2.9%    2817148        proc-vmstat.pgfault
 1.289e+09            -3.5%  1.243e+09        proc-vmstat.pgfree
   2511246            -3.5%    2422977        proc-vmstat.thp_fault_alloc
     88.56           -88.6        0.00        perf-profile.calltrace.cycles-pp.clear_subpage.clear_huge_page.do_huge_pmd_anonymous_page.__handle_mm_fault.handle_mm_fault
     86.61           -86.6        0.00        perf-profile.calltrace.cycles-pp.clear_page_erms.clear_subpage.clear_huge_page.do_huge_pmd_anonymous_page.__handle_mm_fault
     90.95            -0.4       90.57        perf-profile.calltrace.cycles-pp.clear_huge_page.do_huge_pmd_anonymous_page.__handle_mm_fault.handle_mm_fault.do_page_fault
      2.48 ±  3%      -0.2        2.28 ±  2%  perf-profile.calltrace.cycles-pp.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe
      2.37 ±  3%      -0.2        2.17 ±  2%  perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu.tlb_finish_mmu.unmap_region.__do_munmap
      2.52 ±  3%      -0.2        2.31 ±  2%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.munmap
      2.52 ±  3%      -0.2        2.31 ±  2%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap
      2.52 ±  3%      -0.2        2.31 ±  2%  perf-profile.calltrace.cycles-pp.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap
      2.52 ±  3%      -0.2        2.31 ±  2%  perf-profile.calltrace.cycles-pp.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap
      2.46 ±  3%      -0.2        2.25 ±  2%  perf-profile.calltrace.cycles-pp.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64
      2.52 ±  3%      -0.2        2.32 ±  2%  perf-profile.calltrace.cycles-pp.munmap
      2.42 ±  3%      -0.2        2.22 ±  2%  perf-profile.calltrace.cycles-pp.tlb_finish_mmu.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap
      2.42 ±  3%      -0.2        2.22 ±  2%  perf-profile.calltrace.cycles-pp.tlb_flush_mmu.tlb_finish_mmu.unmap_region.__do_munmap.__vm_munmap
      2.25 ±  3%      -0.2        2.04 ±  3%  perf-profile.calltrace.cycles-pp.__free_pages_ok.release_pages.tlb_flush_mmu.tlb_finish_mmu.unmap_region
      3.34            -0.1        3.22        perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.do_huge_pmd_anonymous_page.__handle_mm_fault
      3.36            -0.1        3.25        perf-profile.calltrace.cycles-pp.alloc_pages_vma.do_huge_pmd_anonymous_page.__handle_mm_fault.handle_mm_fault.do_page_fault
      3.35            -0.1        3.24        perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.alloc_pages_vma.do_huge_pmd_anonymous_page.__handle_mm_fault.handle_mm_fault
      0.61 ±  3%      +0.2        0.78 ±  2%  perf-profile.calltrace.cycles-pp._cond_resched.clear_huge_page.do_huge_pmd_anonymous_page.__handle_mm_fault.handle_mm_fault
     95.16            +0.3       95.45        perf-profile.calltrace.cycles-pp.page_fault
     94.98            +0.3       95.28        perf-profile.calltrace.cycles-pp.handle_mm_fault.do_page_fault.page_fault
     95.12            +0.3       95.42        perf-profile.calltrace.cycles-pp.do_page_fault.page_fault
     94.95            +0.3       95.26        perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_page_fault.page_fault
     94.89            +0.3       95.22        perf-profile.calltrace.cycles-pp.do_huge_pmd_anonymous_page.__handle_mm_fault.handle_mm_fault.do_page_fault.page_fault
     88.69           -88.7        0.00        perf-profile.children.cycles-pp.clear_subpage
     87.05           -86.9        0.12 ±  9%  perf-profile.children.cycles-pp.clear_page_erms
      0.88 ±  3%      -0.8        0.10 ± 15%  perf-profile.children.cycles-pp.rcu_all_qs
      2.85 ±  3%      -0.2        2.63 ±  3%  perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
      2.85 ±  3%      -0.2        2.63 ±  3%  perf-profile.children.cycles-pp.do_syscall_64
      2.49 ±  3%      -0.2        2.28 ±  2%  perf-profile.children.cycles-pp.__do_munmap
      2.43 ±  3%      -0.2        2.22 ±  2%  perf-profile.children.cycles-pp.tlb_finish_mmu
      2.52 ±  3%      -0.2        2.31 ±  2%  perf-profile.children.cycles-pp.__x64_sys_munmap
      2.46 ±  3%      -0.2        2.26 ±  2%  perf-profile.children.cycles-pp.unmap_region
      2.43 ±  3%      -0.2        2.22 ±  2%  perf-profile.children.cycles-pp.tlb_flush_mmu
      2.38 ±  3%      -0.2        2.18 ±  2%  perf-profile.children.cycles-pp.release_pages
      2.52 ±  3%      -0.2        2.32 ±  2%  perf-profile.children.cycles-pp.munmap
      2.52 ±  3%      -0.2        2.31 ±  2%  perf-profile.children.cycles-pp.__vm_munmap
      2.25 ±  3%      -0.2        2.05 ±  3%  perf-profile.children.cycles-pp.__free_pages_ok
      3.58            -0.1        3.43        perf-profile.children.cycles-pp.__alloc_pages_nodemask
      3.55            -0.1        3.41        perf-profile.children.cycles-pp.get_page_from_freelist
      0.95 ±  4%      -0.1        0.84 ±  2%  perf-profile.children.cycles-pp._cond_resched
      3.37            -0.1        3.27        perf-profile.children.cycles-pp.alloc_pages_vma
      0.68            -0.1        0.59 ±  2%  perf-profile.children.cycles-pp.apic_timer_interrupt
      0.33 ±  3%      -0.1        0.27 ±  8%  perf-profile.children.cycles-pp.hrtimer_interrupt
      0.44 ±  2%      -0.1        0.37 ±  3%  perf-profile.children.cycles-pp.smp_apic_timer_interrupt
      0.26 ±  3%      -0.1        0.20 ±  9%  perf-profile.children.cycles-pp.__hrtimer_run_queues
      0.19 ±  3%      -0.0        0.14 ±  3%  perf-profile.children.cycles-pp.tick_sched_timer
      0.16 ±  5%      -0.0        0.12 ±  3%  perf-profile.children.cycles-pp.tick_sched_handle
      0.22 ±  5%      -0.0        0.19 ±  6%  perf-profile.children.cycles-pp.pte_alloc_one
      0.15 ±  5%      -0.0        0.12 ±  3%  perf-profile.children.cycles-pp.update_process_times
     95.14            +0.3       95.44        perf-profile.children.cycles-pp.do_page_fault
     95.01            +0.3       95.31        perf-profile.children.cycles-pp.handle_mm_fault
     95.19            +0.3       95.49        perf-profile.children.cycles-pp.page_fault
     94.98            +0.3       95.29        perf-profile.children.cycles-pp.__handle_mm_fault
     94.89            +0.3       95.22        perf-profile.children.cycles-pp.do_huge_pmd_anonymous_page
     91.02            +0.5       91.52        perf-profile.children.cycles-pp.clear_huge_page
     86.50           -86.4        0.12 ±  9%  perf-profile.self.cycles-pp.clear_page_erms
      0.57 ±  4%      -0.5        0.08 ± 13%  perf-profile.self.cycles-pp.rcu_all_qs
      1.26 ±  4%      -0.5        0.78 ±  3%  perf-profile.self.cycles-pp.___might_sleep
      2.16 ±  3%      -0.2        1.96 ±  3%  perf-profile.self.cycles-pp.__free_pages_ok
      2.09            -0.1        2.00        perf-profile.self.cycles-pp.get_page_from_freelist
      0.36 ±  4%      +0.1        0.50 ±  4%  perf-profile.self.cycles-pp._cond_resched
      0.11 ± 11%     +89.6       89.74        perf-profile.self.cycles-pp.clear_huge_page



***************************************************************************************************
lkp-ivb-d02: 4 threads Intel(R) Core(TM) i3-3220 CPU @ 3.30GHz with 8G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/rootfs/tbox_group/test/testcase/ucode:
  gcc-7/performance/x86_64-rhel-7.6/debian-x86_64-20191114.cgz/lkp-ivb-d02/page_fault1/will-it-scale/0x21

commit: 
  8632e9b564 (" hyperv-fixes for 5.7-rc1")
  9266baf8ee ("mm: Optimized hugepage zeroing & copying from user")

8632e9b5645bbc23 9266baf8eea588a78e5a0aa4ccd 
---------------- --------------------------- 
       fail:runs  %reproduction    fail:runs
           |             |             |    
          0:4            2%           0:4     perf-profile.self.cycles-pp.error_entry
         %stddev     %change         %stddev
             \          |                \  
   2061889            -5.3%    1951865        will-it-scale.per_process_ops
   2171644            -5.7%    2048044        will-it-scale.per_thread_ops
      0.23            +2.1%       0.23        will-it-scale.scalability
   1232399            -5.2%    1168571        will-it-scale.time.minor_page_faults
     16.41            -6.3%      15.38        will-it-scale.time.user_time
     57433            -6.5%      53713        will-it-scale.time.voluntary_context_switches
  32371162            -5.1%   30719792        will-it-scale.workload
   1138201 ± 16%     -10.8%    1015835 ± 16%  cpuidle.C1E.usage
    532.00 ± 11%     -22.6%     412.00 ± 20%  slabinfo.skbuff_head_cache.num_objs
     24570            +2.8%      25263        interrupts.CAL:Function_call_interrupts
      1823 ± 20%     +26.7%       2309 ± 11%  interrupts.CPU0.RES:Rescheduling_interrupts
     28931 ± 11%     -15.5%      24434 ± 14%  softirqs.CPU1.SCHED
     14949 ± 11%     +17.2%      17527 ±  6%  softirqs.CPU2.RCU
  14575349            -4.8%   13875471        proc-vmstat.numa_hit
  14575349            -4.8%   13875471        proc-vmstat.numa_local
 1.228e+09            -5.0%  1.167e+09        proc-vmstat.pgalloc_normal
  12186704            -4.7%   11609076        proc-vmstat.pgfault
 1.228e+09            -5.0%  1.167e+09        proc-vmstat.pgfree
   2374618            -5.0%    2256894        proc-vmstat.thp_fault_alloc
      0.00          +2e+11%       2041 ± 82%  sched_debug.cfs_rq:/.MIN_vruntime.avg
      0.00        +7.5e+11%       7472 ± 94%  sched_debug.cfs_rq:/.MIN_vruntime.max
    266202 ±  2%     +16.4%     309784 ±  5%  sched_debug.cfs_rq:/.load.avg
      0.00          +2e+11%       2041 ± 82%  sched_debug.cfs_rq:/.max_vruntime.avg
      0.00        +7.5e+11%       7472 ± 94%  sched_debug.cfs_rq:/.max_vruntime.max
    155407 ±  7%     -19.4%     125275 ± 15%  sched_debug.cfs_rq:/.min_vruntime.min
     58058 ± 16%     +28.5%      74586 ± 10%  sched_debug.cfs_rq:/.min_vruntime.stddev
      4.17 ± 36%     +46.0%       6.08 ± 17%  sched_debug.cfs_rq:/.nr_spread_over.max
     22843 ±100%    +333.2%      98964 ± 51%  sched_debug.cfs_rq:/.spread0.max
     58060 ± 16%     +28.5%      74587 ± 10%  sched_debug.cfs_rq:/.spread0.stddev
     58.60 ±  8%     -58.6        0.00        perf-profile.calltrace.cycles-pp.clear_subpage.clear_huge_page.do_huge_pmd_anonymous_page.__handle_mm_fault.handle_mm_fault
     57.50 ±  8%     -57.5        0.00        perf-profile.calltrace.cycles-pp.clear_page_erms.clear_subpage.clear_huge_page.do_huge_pmd_anonymous_page.__handle_mm_fault
      0.67 ±  4%      -0.1        0.59 ±  6%  perf-profile.calltrace.cycles-pp.prep_new_page.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.handle_pte_fault
      0.65 ±  3%      -0.1        0.57 ±  7%  perf-profile.calltrace.cycles-pp.clear_page_erms.prep_new_page.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma
     58.68 ±  8%     -58.7        0.00        perf-profile.children.cycles-pp.clear_subpage
     58.41 ±  8%     -57.7        0.69 ±  7%  perf-profile.children.cycles-pp.clear_page_erms
      0.08 ±  8%      -0.0        0.06 ±  6%  perf-profile.children.cycles-pp.free_pages_and_swap_cache
      0.10 ± 11%      +0.0        0.13 ±  3%  perf-profile.children.cycles-pp._raw_spin_lock
      0.00            +0.1        0.09 ± 29%  perf-profile.children.cycles-pp.forkshell
      0.23 ± 19%      +0.3        0.50 ±  5%  perf-profile.children.cycles-pp.rcu_all_qs
     57.92 ±  8%     -57.2        0.69 ±  6%  perf-profile.self.cycles-pp.clear_page_erms
      0.08 ±  5%      -0.0        0.06 ±  6%  perf-profile.self.cycles-pp.free_pages_and_swap_cache
      0.18 ± 13%      +0.1        0.26 ±  7%  perf-profile.self.cycles-pp.rcu_all_qs
      0.72 ± 13%     +60.5       61.22 ±  6%  perf-profile.self.cycles-pp.clear_huge_page





Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


Thanks,
Rong Chen


View attachment "config-5.7.0-rc1-00025-g9266baf8eea588" of type "text/plain" (206562 bytes)

View attachment "job-script" of type "text/plain" (7723 bytes)

View attachment "job.yaml" of type "text/plain" (5355 bytes)

View attachment "reproduce" of type "text/plain" (14027 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ