lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <20210113024605.GB7528@xsang-OptiPlex-9020>
Date:   Wed, 13 Jan 2021 10:46:05 +0800
From:   kernel test robot <oliver.sang@...el.com>
To:     Frederic Weisbecker <frederic@...nel.org>
Cc:     0day robot <lkp@...el.com>, Peter Zijlstra <peterz@...radead.org>,
        Thomas Gleixner <tglx@...utronix.de>,
        "Paul E. McKenney" <paulmck@...nel.org>,
        "Rafael J. Wysocki" <rafael.j.wysocki@...el.com>,
        LKML <linux-kernel@...r.kernel.org>, lkp@...ts.01.org,
        ying.huang@...el.com, feng.tang@...el.com, zhengjun.xing@...el.com,
        Frederic Weisbecker <frederic@...nel.org>,
        Ingo Molnar <mingo@...nel.org>, stable@...r.kernel.org
Subject: [entry]  8e01c5f104:  unixbench.score -2.2% regression


Greeting,

FYI, we noticed a -2.2% regression of unixbench.score due to commit:


commit: 8e01c5f10451c019e384d68ee8edb9129e3f0f7f ("entry: Report local wake up on resched blind zone while resuming to user")
url: https://github.com/0day-ci/linux/commits/Frederic-Weisbecker/rcu-sched-Fix-ignored-rescheduling-after-rcu_eqs_enter-v3/20210109-100950


in testcase: unixbench
on test machine: 96 threads Intel(R) Xeon(R) CPU @ 2.30GHz with 128G memory
with following parameters:

	runtime: 300s
	nr_task: 1
	test: syscall
	cpufreq_governor: performance
	ucode: 0x4003003

test-description: UnixBench is the original BYTE UNIX benchmark suite aims to test performance of Unix-like system.
test-url: https://github.com/kdlucas/byte-unixbench

In addition to that, the commit also has significant impact on the following tests:

+------------------+---------------------------------------------------------------------------+
| testcase: change | will-it-scale: will-it-scale.per_thread_ops -2.0% regression              |
| test machine     | 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory |
| test parameters  | cpufreq_governor=performance                                              |
|                  | mode=thread                                                               |
|                  | nr_task=50%                                                               |
|                  | test=futex3                                                               |
|                  | ucode=0x5003003                                                           |
+------------------+---------------------------------------------------------------------------+
| testcase: change | will-it-scale: will-it-scale.per_thread_ops -1.5% regression              |
| test machine     | 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory |
| test parameters  | cpufreq_governor=performance                                              |
|                  | mode=thread                                                               |
|                  | nr_task=16                                                                |
|                  | test=futex4                                                               |
|                  | ucode=0x5003003                                                           |
+------------------+---------------------------------------------------------------------------+


If you fix the issue, kindly add following tag
Reported-by: kernel test robot <oliver.sang@...el.com>


Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp install job.yaml  # job file is attached in this email
        bin/lkp run     job.yaml

=========================================================================================
compiler/cpufreq_governor/kconfig/nr_task/rootfs/runtime/tbox_group/test/testcase/ucode:
  gcc-9/performance/x86_64-rhel-8.3/1/debian-10.4-x86_64-20200603.cgz/300s/lkp-csl-2sp4/syscall/unixbench/0x4003003

commit: 
  9720a64438 ("sched: Report local wake up on resched blind zone within idle loop")
  8e01c5f104 ("entry: Report local wake up on resched blind zone while resuming to user")

9720a64438d901da 8e01c5f10451c019e384d68ee8e 
---------------- --------------------------- 
       fail:runs  %reproduction    fail:runs
           |             |             |    
          0:4           -2%           0:4     perf-profile.children.cycles-pp.error_entry
          0:4           -1%           0:4     perf-profile.self.cycles-pp.error_entry
         %stddev     %change         %stddev
             \          |                \  
      1566            -2.2%       1532        unixbench.score
    198.20            -1.2%     195.82        unixbench.time.system_time
    100.35            +2.4%     102.77        unixbench.time.user_time
 9.165e+08            -2.2%  8.965e+08        unixbench.workload
    105519 ±116%     -72.3%      29231 ± 10%  cpuidle.C1.usage
      0.02 ± 31%     -56.9%       0.01 ± 33%  perf-sched.sch_delay.max.ms.schedule_timeout.wait_for_completion.__flush_work.lru_add_drain_all
     10909 ±  4%     -12.2%       9580 ±  6%  numa-vmstat.node0.nr_slab_reclaimable
      7745 ±  5%     +17.3%       9087 ±  8%  numa-vmstat.node1.nr_slab_reclaimable
      2558 ±  5%     +16.4%       2977        slabinfo.fsnotify_mark_connector.active_objs
      2558 ±  5%     +16.4%       2977        slabinfo.fsnotify_mark_connector.num_objs
    570484 ±  4%      +6.7%     608647 ±  6%  sched_debug.cpu.max_idle_balance_cost.max
     10507 ± 42%     +62.3%      17056 ± 11%  sched_debug.cpu.max_idle_balance_cost.stddev
      8.73 ±  7%     -16.0%       7.33 ±  5%  sched_debug.cpu.nr_uninterruptible.stddev
     43640 ±  4%     -12.2%      38321 ±  6%  numa-meminfo.node0.KReclaimable
     43640 ±  4%     -12.2%      38321 ±  6%  numa-meminfo.node0.SReclaimable
    135268 ±  2%      -8.5%     123810 ±  4%  numa-meminfo.node0.Slab
     30984 ±  5%     +17.3%      36352 ±  8%  numa-meminfo.node1.KReclaimable
     30984 ±  5%     +17.3%      36352 ±  8%  numa-meminfo.node1.SReclaimable
    101801 ±  3%     +11.6%     113655 ±  4%  numa-meminfo.node1.Slab
 7.036e+08 ±  2%      +4.3%   7.34e+08        perf-stat.i.branch-instructions
 1.074e+09            +2.5%  1.101e+09        perf-stat.i.dTLB-loads
 6.915e+08            +4.1%  7.199e+08        perf-stat.i.dTLB-stores
     26.16            +3.0%      26.93        perf-stat.i.metric.M/sec
      1479 ±  2%      +4.1%       1540        perf-stat.overall.path-length
 7.018e+08 ±  2%      +4.3%  7.322e+08        perf-stat.ps.branch-instructions
 1.071e+09            +2.6%  1.098e+09        perf-stat.ps.dTLB-loads
 6.895e+08            +4.1%  7.179e+08        perf-stat.ps.dTLB-stores
      3.75 ±  5%      -0.8        2.99 ± 15%  perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack
      2.99 ±  6%      -0.6        2.39 ± 17%  perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt
      1.46 ±  6%      -0.3        1.18 ± 14%  perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
      0.96 ± 10%      +0.2        1.16 ± 12%  perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe
      3.86 ±  4%      -0.8        3.06 ± 15%  perf-profile.children.cycles-pp.tick_sched_timer
      3.09 ±  6%      -0.6        2.48 ± 16%  perf-profile.children.cycles-pp.update_process_times
      1.51 ±  6%      -0.3        1.25 ± 13%  perf-profile.children.cycles-pp.scheduler_tick
      0.05 ± 58%      +0.0        0.09 ± 12%  perf-profile.children.cycles-pp.rcu_dynticks_eqs_enter
      0.28 ± 11%      +0.1        0.34 ±  7%  perf-profile.children.cycles-pp.__intel_pmu_enable_all
      0.93 ±  7%      +0.1        1.07 ± 12%  perf-profile.children.cycles-pp.syscall_enter_from_user_mode
      0.03 ±100%      +0.2        0.18 ± 17%  perf-profile.children.cycles-pp.sched_resched_local_allow
      1.47 ±  8%      +0.3        1.75 ± 10%  perf-profile.children.cycles-pp.exit_to_user_mode_prepare
      0.00            +0.3        0.33 ± 10%  perf-profile.children.cycles-pp.sched_resched_local_forbid
      0.47 ±  9%      -0.1        0.36 ± 19%  perf-profile.self.cycles-pp.update_process_times
      0.05 ± 58%      +0.0        0.09 ± 12%  perf-profile.self.cycles-pp.rcu_dynticks_eqs_enter
      0.10 ±  5%      +0.0        0.14 ± 17%  perf-profile.self.cycles-pp.__x64_sys_close
      0.28 ± 11%      +0.1        0.34 ±  7%  perf-profile.self.cycles-pp.__intel_pmu_enable_all
      0.01 ±173%      +0.2        0.18 ± 15%  perf-profile.self.cycles-pp.sched_resched_local_allow
      0.00            +0.2        0.17 ± 21%  perf-profile.self.cycles-pp.sched_resched_local_forbid
      3.78 ± 48%      +1.4        5.16 ± 39%  perf-profile.self.cycles-pp.cpuidle_enter_state
     75783 ±  2%      +7.7%      81634 ±  3%  interrupts.CAL:Function_call_interrupts
    148.75 ± 14%     -33.4%      99.00 ± 34%  interrupts.CPU16.NMI:Non-maskable_interrupts
    148.75 ± 14%     -33.4%      99.00 ± 34%  interrupts.CPU16.PMI:Performance_monitoring_interrupts
    805.75 ±144%     -87.4%     101.50 ± 33%  interrupts.CPU19.NMI:Non-maskable_interrupts
    805.75 ±144%     -87.4%     101.50 ± 33%  interrupts.CPU19.PMI:Performance_monitoring_interrupts
      1312 ±153%     -92.4%     100.25 ± 34%  interrupts.CPU23.NMI:Non-maskable_interrupts
      1312 ±153%     -92.4%     100.25 ± 34%  interrupts.CPU23.PMI:Performance_monitoring_interrupts
    618.00 ±  5%     +10.3%     681.50 ±  2%  interrupts.CPU39.CAL:Function_call_interrupts
    579.50 ± 12%     +18.2%     685.00 ±  2%  interrupts.CPU48.CAL:Function_call_interrupts
    254.50 ± 65%     -60.8%      99.75 ± 34%  interrupts.CPU48.NMI:Non-maskable_interrupts
    254.50 ± 65%     -60.8%      99.75 ± 34%  interrupts.CPU48.PMI:Performance_monitoring_interrupts
    136.25 ± 13%     -32.5%      92.00 ± 18%  interrupts.CPU49.NMI:Non-maskable_interrupts
    136.25 ± 13%     -32.5%      92.00 ± 18%  interrupts.CPU49.PMI:Performance_monitoring_interrupts
    134.50 ± 15%     -29.9%      94.25 ± 22%  interrupts.CPU50.NMI:Non-maskable_interrupts
    134.50 ± 15%     -29.9%      94.25 ± 22%  interrupts.CPU50.PMI:Performance_monitoring_interrupts
    668.75 ±  5%    +176.1%       1846 ± 64%  interrupts.CPU56.CAL:Function_call_interrupts
    143.50 ± 14%     -23.7%     109.50 ± 15%  interrupts.CPU60.NMI:Non-maskable_interrupts
    143.50 ± 14%     -23.7%     109.50 ± 15%  interrupts.CPU60.PMI:Performance_monitoring_interrupts
    140.75 ± 17%     -32.9%      94.50 ± 26%  interrupts.CPU62.NMI:Non-maskable_interrupts
    140.75 ± 17%     -32.9%      94.50 ± 26%  interrupts.CPU62.PMI:Performance_monitoring_interrupts
    143.00 ± 10%     -43.7%      80.50 ± 36%  interrupts.CPU64.NMI:Non-maskable_interrupts
    143.00 ± 10%     -43.7%      80.50 ± 36%  interrupts.CPU64.PMI:Performance_monitoring_interrupts
    650.75           +20.1%     781.50 ± 20%  interrupts.CPU69.CAL:Function_call_interrupts
    510.00 ±123%     -80.8%      98.00 ± 34%  interrupts.CPU71.NMI:Non-maskable_interrupts
    510.00 ±123%     -80.8%      98.00 ± 34%  interrupts.CPU71.PMI:Performance_monitoring_interrupts
    648.00 ±  2%     +35.6%     878.75 ± 36%  interrupts.CPU73.CAL:Function_call_interrupts
    648.75 ±  2%    +169.4%       1748 ± 92%  interrupts.CPU88.CAL:Function_call_interrupts


                                                                                
                                  unixbench.score                               
                                                                                
  1590 +--------------------------------------------------------------------+   
       |. +. .+. +.+. .+ .+.++   +.+ .+.   +. .+ .+. .++. .+ .+. .+         |   
  1580 |-+  +   +    +  +     :  :  +   +.+  +  +   +    +  +   +  +.+      |   
       |                      : :                                     :     |   
       |                       ::                                     :     |   
  1570 |-+                     +                                       +.+ .|   
       |                                                                  + |   
  1560 |-+                                                                  |   
       |                                                                    |   
  1550 |-+                                                                  |   
       |                                                                    |   
       |                                                                    |   
  1540 |-+                                                                  |   
       | OO O O OO     O     O        O O O                                 |   
  1530 +--------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                  unixbench.workload                            
                                                                                
   9.3e+08 +----------------------------------------------------------------+   
           |.++.+.+ + +.+.++.+.++  +.+.++. .++.++.+.+ +  ++.++.+. : +.      |   
  9.25e+08 |-+     +             : :      +          +           +    +     |   
           |                     : :                                  :     |   
   9.2e+08 |-+                    :                                    :    |   
           |                      +                                    +.+ .|   
  9.15e+08 |-+                                                            + |   
           |                                                                |   
   9.1e+08 |-+                                                              |   
           |                                                                |   
  9.05e+08 |-+                                                              |   
           |                                                                |   
     9e+08 |-+                                                              |   
           | OO O OO OO   OO   OO O  O  O O O                               |   
  8.95e+08 +----------------------------------------------------------------+   
                                                                                
                                                                                
[*] bisect-good sample
[O] bisect-bad  sample

***************************************************************************************************
lkp-csl-2ap2: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode:
  gcc-9/performance/x86_64-rhel-8.3/thread/50%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2ap2/futex3/will-it-scale/0x5003003

commit: 
  9720a64438 ("sched: Report local wake up on resched blind zone within idle loop")
  8e01c5f104 ("entry: Report local wake up on resched blind zone while resuming to user")

9720a64438d901da 8e01c5f10451c019e384d68ee8e 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
 9.783e+08            -2.0%   9.59e+08        will-it-scale.96.threads
  10190429            -2.0%    9989144        will-it-scale.per_thread_ops
 9.783e+08            -2.0%   9.59e+08        will-it-scale.workload
      0.06            +0.0        0.07 ±  2%  mpstat.cpu.all.soft%
     28015            +1.1%      28324        proc-vmstat.nr_slab_reclaimable
      4971 ±  6%     -11.4%       4405 ±  7%  sched_debug.cpu.nr_switches.stddev
      1275 ± 70%    +306.7%       5187 ± 86%  numa-vmstat.node0.nr_shmem
     65283 ±  3%     -17.2%      54026 ± 18%  numa-vmstat.node3.nr_shmem
      2721 ±  3%     +12.1%       3049 ±  4%  slabinfo.PING.active_objs
      2721 ±  3%     +12.1%       3049 ±  4%  slabinfo.PING.num_objs
      1520 ±  6%     +17.8%       1790 ±  7%  slabinfo.khugepaged_mm_slot.active_objs
      1520 ±  6%     +17.8%       1790 ±  7%  slabinfo.khugepaged_mm_slot.num_objs
      5105 ± 70%    +307.4%      20798 ± 86%  numa-meminfo.node0.Shmem
    372490 ± 36%     -57.6%     157918 ± 53%  numa-meminfo.node1.AnonPages.max
    251355 ±  3%     -17.6%     207138 ± 18%  numa-meminfo.node3.Active
    251355 ±  3%     -17.6%     207138 ± 18%  numa-meminfo.node3.Active(anon)
    261667 ±  3%     -17.3%     216523 ± 18%  numa-meminfo.node3.Shmem
    946.63 ±173%    +493.3%       5616 ± 26%  perf-sched.wait_and_delay.avg.ms.preempt_schedule_common._cond_resched.generic_perform_write.__generic_file_write_iter.generic_file_write_iter
    240.00 ± 48%     -36.7%     152.00 ± 60%  perf-sched.wait_and_delay.count.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown]
    148.50 ± 17%     -24.1%     112.75 ± 13%  perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
      1873 ±173%    +300.6%       7504        perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.generic_perform_write.__generic_file_write_iter.generic_file_write_iter
      0.02 ± 39%     -76.9%       0.00 ±173%  perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single.[unknown]
    973.25 ±166%    +477.1%       5616 ± 26%  perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.generic_perform_write.__generic_file_write_iter.generic_file_write_iter
      0.03 ± 41%     -74.1%       0.01 ±173%  perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single.[unknown]
      2031 ±155%    +269.4%       7504        perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.generic_perform_write.__generic_file_write_iter.generic_file_write_iter
      0.01 ± 60%    +133.3%       0.02 ± 19%  perf-sched.wait_time.max.ms.schedule_timeout.wait_for_completion.stop_one_cpu.affine_move_task
 6.958e+10            +3.6%  7.205e+10        perf-stat.i.branch-instructions
      0.72            -0.0        0.68        perf-stat.i.branch-miss-rate%
 4.961e+08            -1.9%  4.867e+08        perf-stat.i.branch-misses
     14.70 ±  3%      +1.1       15.81        perf-stat.i.cache-miss-rate%
   1497135 ±  4%     +11.5%    1668752 ±  4%  perf-stat.i.cache-misses
    228415 ±  4%     -11.6%     201875 ±  6%  perf-stat.i.cycles-between-cache-misses
 1.114e+11            +1.5%  1.131e+11        perf-stat.i.dTLB-loads
 8.403e+10            +2.6%  8.619e+10        perf-stat.i.dTLB-stores
   3747984            +2.7%    3849820        perf-stat.i.iTLB-loads
      1.53 ±  4%      +5.8%       1.62 ±  3%  perf-stat.i.major-faults
      1.39            +5.1%       1.46 ±  3%  perf-stat.i.metric.K/sec
      1379            +2.4%       1412        perf-stat.i.metric.M/sec
    301494            +9.0%     328692 ±  5%  perf-stat.i.node-load-misses
      0.71            -0.0        0.68        perf-stat.overall.branch-miss-rate%
     14.61 ±  3%      +0.9       15.55        perf-stat.overall.cache-miss-rate%
    195763 ±  4%     -10.5%     175161 ±  4%  perf-stat.overall.cycles-between-cache-misses
      0.00            -0.0        0.00        perf-stat.overall.dTLB-store-miss-rate%
    134378            +2.2%     137315        perf-stat.overall.path-length
  6.93e+10            +3.5%  7.175e+10        perf-stat.ps.branch-instructions
 4.942e+08            -1.9%  4.848e+08        perf-stat.ps.branch-misses
   1510988 ±  4%     +11.4%    1683127 ±  4%  perf-stat.ps.cache-misses
    203.58            -1.5%     200.43        perf-stat.ps.cpu-migrations
  1.11e+11            +1.5%  1.126e+11        perf-stat.ps.dTLB-loads
 8.368e+10            +2.6%  8.583e+10        perf-stat.ps.dTLB-stores
   3733148            +2.7%    3832271        perf-stat.ps.iTLB-loads
    305850            +9.2%     333869 ±  5%  perf-stat.ps.node-load-misses
      1.52 ± 10%      +0.3        1.79 ± 11%  perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe.syscall
      1.68 ±  9%      +0.3        2.01 ± 11%  perf-profile.calltrace.cycles-pp.syscall_enter_from_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.syscall
      3.23 ± 10%      +0.4        3.58 ± 11%  perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe.syscall
      0.10 ± 23%      -0.1        0.04 ± 58%  perf-profile.children.cycles-pp.ktime_get
      0.09 ± 14%      -0.0        0.04 ± 59%  perf-profile.children.cycles-pp.clockevents_program_event
      0.09 ± 10%      +0.0        0.13 ±  9%  perf-profile.children.cycles-pp.perf_prepare_sample
      0.11 ±  8%      +0.0        0.15 ±  8%  perf-profile.children.cycles-pp.perf_tp_event
      0.10 ± 10%      +0.0        0.15 ± 10%  perf-profile.children.cycles-pp.perf_swevent_overflow
      0.11 ±  8%      +0.0        0.15 ± 10%  perf-profile.children.cycles-pp.perf_trace_sched_stat_runtime
      0.10 ± 12%      +0.1        0.15 ± 10%  perf-profile.children.cycles-pp.__perf_event_overflow
      0.10 ± 12%      +0.1        0.15 ± 10%  perf-profile.children.cycles-pp.perf_event_output_forward
      0.00            +0.1        0.06 ± 14%  perf-profile.children.cycles-pp.account_system_index_time
      0.20 ± 10%      +0.1        0.26 ±  9%  perf-profile.children.cycles-pp.task_tick_fair
      0.11 ± 11%      +0.1        0.18 ± 10%  perf-profile.children.cycles-pp.update_curr
      0.22 ±  9%      +0.1        0.29 ±  9%  perf-profile.children.cycles-pp.scheduler_tick
      0.35 ±  7%      +0.1        0.47 ±  9%  perf-profile.children.cycles-pp.__hrtimer_run_queues
      0.30 ± 10%      +0.1        0.43 ±  9%  perf-profile.children.cycles-pp.tick_sched_timer
      0.28 ±  9%      +0.1        0.42 ±  9%  perf-profile.children.cycles-pp.update_process_times
      0.28 ±  9%      +0.1        0.43 ±  8%  perf-profile.children.cycles-pp.tick_sched_handle
      0.00            +0.2        0.22 ± 11%  perf-profile.children.cycles-pp.sched_resched_local_allow
      2.37 ± 10%      +0.2        2.61 ± 12%  perf-profile.children.cycles-pp.testcase
      1.94 ± 10%      +0.3        2.23 ± 11%  perf-profile.children.cycles-pp.exit_to_user_mode_prepare
      1.69 ±  9%      +0.3        2.02 ± 11%  perf-profile.children.cycles-pp.syscall_enter_from_user_mode
      3.66 ± 10%      +0.4        4.02 ± 11%  perf-profile.children.cycles-pp.syscall_exit_to_user_mode
      0.00            +0.4        0.45 ± 11%  perf-profile.children.cycles-pp.sched_resched_local_forbid
      0.09 ± 20%      -0.1        0.03 ±100%  perf-profile.self.cycles-pp.ktime_get
      0.00            +0.1        0.05 ±  9%  perf-profile.self.cycles-pp.account_system_index_time
      1.91 ± 10%      +0.2        2.13 ± 12%  perf-profile.self.cycles-pp.testcase
      0.00            +0.2        0.22 ± 11%  perf-profile.self.cycles-pp.sched_resched_local_forbid
      0.00            +0.2        0.22 ± 11%  perf-profile.self.cycles-pp.sched_resched_local_allow
     39568           -12.1%      34775        softirqs.CPU0.SCHED
     26074 ±  6%     -30.8%      18054 ± 18%  softirqs.CPU1.RCU
     13937 ± 27%     +96.1%      27328 ± 20%  softirqs.CPU1.SCHED
    487.75 ± 60%   +1455.7%       7587 ±129%  softirqs.CPU10.NET_RX
      7471 ± 99%     -99.6%      32.50 ± 38%  softirqs.CPU103.TIMER
     22133 ± 15%     -27.0%      16160 ± 29%  softirqs.CPU107.RCU
     23683 ± 12%     -34.9%      15423 ± 25%  softirqs.CPU110.RCU
     21771 ± 13%     -27.0%      15887 ± 28%  softirqs.CPU119.RCU
     27268 ±  7%     -33.6%      18105 ± 24%  softirqs.CPU12.RCU
      9800 ± 82%    +147.2%      24228 ± 16%  softirqs.CPU12.SCHED
     35848 ± 10%     -52.3%      17101 ± 52%  softirqs.CPU123.SCHED
     21873 ±  9%     -28.4%      15658 ± 19%  softirqs.CPU125.RCU
     23701 ±  7%     -24.4%      17906 ± 20%  softirqs.CPU129.RCU
     23812 ± 15%     -27.5%      17268 ±  7%  softirqs.CPU130.RCU
     35487 ±  8%     -38.9%      21674 ± 33%  softirqs.CPU131.SCHED
     24202 ± 14%     -26.3%      17841 ± 24%  softirqs.CPU139.RCU
     26857 ±  9%     -33.1%      17956 ± 24%  softirqs.CPU145.RCU
     24985           -25.4%      18643 ± 24%  softirqs.CPU146.RCU
     19845 ± 11%     +32.6%      26307 ± 18%  softirqs.CPU146.SCHED
     24163 ± 10%     -30.7%      16746 ± 16%  softirqs.CPU147.RCU
     25991 ± 11%     -28.0%      18706 ± 20%  softirqs.CPU150.RCU
     31382 ± 16%     -46.1%      16909 ± 33%  softirqs.CPU156.SCHED
     26315 ±  5%     -29.0%      18686 ± 28%  softirqs.CPU16.RCU
     24924 ±  9%     -26.4%      18336 ± 26%  softirqs.CPU163.RCU
     25795 ± 12%     -30.4%      17948 ± 17%  softirqs.CPU165.RCU
     23494 ±  9%     -31.4%      16118 ± 17%  softirqs.CPU169.RCU
     15434 ± 38%     +67.3%      25820 ± 21%  softirqs.CPU169.SCHED
     23443 ±  7%     -25.3%      17521 ± 15%  softirqs.CPU17.RCU
     22698 ±  9%     -20.2%      18116 ± 15%  softirqs.CPU172.RCU
     21677 ±  9%     -29.8%      15224 ± 15%  softirqs.CPU173.RCU
     20602 ± 27%     +53.8%      31690 ± 16%  softirqs.CPU173.SCHED
     19982 ± 10%     -23.1%      15368 ± 16%  softirqs.CPU188.RCU
     31405 ±  9%     -52.0%      15062 ± 43%  softirqs.CPU189.SCHED
     27459 ±  5%     -29.9%      19244 ± 23%  softirqs.CPU19.RCU
     23837 ±  8%     -24.8%      17931 ± 21%  softirqs.CPU191.RCU
     27482 ±  3%     -26.7%      20133 ± 25%  softirqs.CPU2.RCU
     27374 ±  5%     -28.3%      19620 ± 29%  softirqs.CPU20.RCU
      8946 ± 55%    +120.5%      19723 ± 40%  softirqs.CPU20.SCHED
     23561 ±  9%     -27.4%      17102 ± 18%  softirqs.CPU21.RCU
     24920 ±  8%     -28.3%      17869 ± 14%  softirqs.CPU22.RCU
     27899 ±  5%     -36.3%      17760 ± 29%  softirqs.CPU27.RCU
      9230 ± 33%    +202.9%      27954 ± 31%  softirqs.CPU27.SCHED
     25209 ±  7%     -24.7%      18973 ± 22%  softirqs.CPU3.RCU
     27974 ±  9%     -31.3%      19231 ± 13%  softirqs.CPU32.RCU
     28747 ±  5%     -36.5%      18268 ± 14%  softirqs.CPU35.RCU
      9574 ± 33%    +145.3%      23490 ± 32%  softirqs.CPU35.SCHED
     24738 ± 15%     -27.4%      17967 ± 15%  softirqs.CPU36.RCU
     27437 ± 13%     -34.7%      17904 ± 22%  softirqs.CPU37.RCU
     27259 ±  9%     -33.7%      18083 ± 23%  softirqs.CPU38.RCU
     14438 ± 52%     +86.8%      26971 ± 13%  softirqs.CPU38.SCHED
     26156 ±  9%     -32.6%      17617 ± 29%  softirqs.CPU4.RCU
     27287 ±  6%     -31.5%      18695 ± 27%  softirqs.CPU40.RCU
     26370 ± 10%     -30.6%      18302 ± 17%  softirqs.CPU41.RCU
     26793 ±  8%     -30.3%      18668 ± 19%  softirqs.CPU46.RCU
     15557 ± 45%     +64.8%      25642 ± 21%  softirqs.CPU46.SCHED
     25335 ± 12%     -27.3%      18416 ± 24%  softirqs.CPU47.RCU
     25154 ±  2%     -25.0%      18872 ± 20%  softirqs.CPU5.RCU
     23480 ±  4%     -23.3%      18018 ± 23%  softirqs.CPU55.RCU
     26294 ±  3%     -33.0%      17630 ± 20%  softirqs.CPU56.RCU
     13958 ± 32%    +109.1%      29187 ± 15%  softirqs.CPU56.SCHED
     27194 ±  7%     -32.8%      18287 ± 22%  softirqs.CPU57.RCU
     26424 ±  7%     -33.4%      17603 ± 23%  softirqs.CPU60.RCU
     13405 ± 41%    +110.0%      28152 ± 20%  softirqs.CPU60.SCHED
     24662 ± 17%     -30.3%      17187 ± 32%  softirqs.CPU66.RCU
     27174 ± 28%     -33.1%      18168 ± 48%  softirqs.CPU67.SCHED
     23980 ±  7%     -28.8%      17083 ± 25%  softirqs.CPU7.RCU
     16015 ± 12%     +69.5%      27140 ± 30%  softirqs.CPU7.SCHED
     29430 ± 19%     -35.8%      18884 ± 34%  softirqs.CPU73.SCHED
     25123 ±  6%     -25.5%      18715 ± 18%  softirqs.CPU74.RCU
     24340 ± 22%     -44.1%      13615 ± 38%  softirqs.CPU77.SCHED
     23940 ±  8%     -24.2%      18145 ± 22%  softirqs.CPU83.RCU
     22452 ±  7%     -18.7%      18253 ± 15%  softirqs.CPU90.RCU
     24046 ±  3%     -32.2%      16309 ± 24%  softirqs.CPU93.RCU
     13685 ± 19%    +119.3%      30012 ± 21%  softirqs.CPU93.SCHED
      9316 ±  5%     +40.3%      13075 ±  3%  softirqs.CPU96.SCHED
     32207 ±  9%     -45.1%      17687 ± 30%  softirqs.CPU97.SCHED
     37350 ±  4%     -24.4%      28241 ± 21%  softirqs.CPU98.SCHED
     30743 ± 11%     -22.5%      23841 ± 14%  softirqs.CPU99.SCHED
    932.00 ± 64%   +1402.0%      13998 ±133%  interrupts.31:PCI-MSI.524289-edge.eth0-TxRx-0
    120.75 ±  8%     +75.6%     212.00 ±  5%  interrupts.CPU0.RES:Rescheduling_interrupts
    223.00 ± 11%     -64.6%      79.00 ± 38%  interrupts.CPU1.RES:Rescheduling_interrupts
    981.50 ± 19%     -42.4%     565.25 ± 46%  interrupts.CPU1.TLB:TLB_shootdowns
    932.00 ± 64%   +1402.0%      13998 ±133%  interrupts.CPU10.31:PCI-MSI.524289-edge.eth0-TxRx-0
      4140 ± 28%     +84.1%       7623 ± 14%  interrupts.CPU100.NMI:Non-maskable_interrupts
      4140 ± 28%     +84.1%       7623 ± 14%  interrupts.CPU100.PMI:Performance_monitoring_interrupts
      3585 ±  8%      -9.7%       3238 ±  5%  interrupts.CPU104.CAL:Function_call_interrupts
      6437 ± 14%     +34.5%       8655        interrupts.CPU104.NMI:Non-maskable_interrupts
      6437 ± 14%     +34.5%       8655        interrupts.CPU104.PMI:Performance_monitoring_interrupts
     49.50 ±129%    +169.2%     133.25 ± 41%  interrupts.CPU108.RES:Rescheduling_interrupts
    276.75 ±108%    +237.5%     934.00 ± 38%  interrupts.CPU108.TLB:TLB_shootdowns
      3058 ± 11%     +15.2%       3523 ±  5%  interrupts.CPU11.CAL:Function_call_interrupts
      8162 ± 12%     -38.5%       5023 ± 47%  interrupts.CPU110.NMI:Non-maskable_interrupts
      8162 ± 12%     -38.5%       5023 ± 47%  interrupts.CPU110.PMI:Performance_monitoring_interrupts
      3115 ±  6%     -18.7%       2534 ±  4%  interrupts.CPU114.CAL:Function_call_interrupts
     32.25 ±113%    +271.3%     119.75 ± 58%  interrupts.CPU116.RES:Rescheduling_interrupts
      3704 ±  2%     -18.4%       3021 ± 14%  interrupts.CPU12.CAL:Function_call_interrupts
      1544 ±  6%     -51.3%     752.00 ± 48%  interrupts.CPU12.TLB:TLB_shootdowns
      2530 ± 10%     +57.7%       3991 ± 15%  interrupts.CPU123.CAL:Function_call_interrupts
     34.75 ± 80%    +415.1%     179.00 ± 35%  interrupts.CPU123.RES:Rescheduling_interrupts
    264.75 ± 60%    +330.3%       1139 ± 42%  interrupts.CPU123.TLB:TLB_shootdowns
      8062 ±  9%     -29.0%       5722 ± 35%  interrupts.CPU125.NMI:Non-maskable_interrupts
      8062 ±  9%     -29.0%       5722 ± 35%  interrupts.CPU125.PMI:Performance_monitoring_interrupts
      1059 ± 24%     -51.3%     515.75 ± 51%  interrupts.CPU125.TLB:TLB_shootdowns
      2648 ± 12%     +37.0%       3627 ±  9%  interrupts.CPU131.CAL:Function_call_interrupts
     35.75 ± 76%    +253.1%     126.25 ± 32%  interrupts.CPU131.RES:Rescheduling_interrupts
    426.00 ± 44%    +148.5%       1058 ± 27%  interrupts.CPU131.TLB:TLB_shootdowns
    737.50 ± 44%     +60.4%       1182 ± 13%  interrupts.CPU133.TLB:TLB_shootdowns
     76.50 ± 77%    +104.2%     156.25 ± 30%  interrupts.CPU134.RES:Rescheduling_interrupts
    568.25 ± 55%     +62.7%     924.75 ± 30%  interrupts.CPU134.TLB:TLB_shootdowns
      2879 ±  4%     +14.7%       3303 ±  4%  interrupts.CPU136.CAL:Function_call_interrupts
    484.00 ± 66%    +114.2%       1036 ± 20%  interrupts.CPU136.TLB:TLB_shootdowns
     82.25 ± 69%     +88.8%     155.25 ± 30%  interrupts.CPU142.RES:Rescheduling_interrupts
      4178 ± 17%     -24.9%       3136 ± 13%  interrupts.CPU145.CAL:Function_call_interrupts
    204.00 ± 35%     -66.8%      67.75 ± 25%  interrupts.CPU145.RES:Rescheduling_interrupts
      1429 ± 17%     -49.8%     717.50 ± 32%  interrupts.CPU145.TLB:TLB_shootdowns
    165.50 ±  9%     -45.9%      89.50 ± 23%  interrupts.CPU146.RES:Rescheduling_interrupts
      8063 ± 14%     -53.9%       3717 ± 15%  interrupts.CPU15.NMI:Non-maskable_interrupts
      8063 ± 14%     -53.9%       3717 ± 15%  interrupts.CPU15.PMI:Performance_monitoring_interrupts
      2702 ±  4%     +23.8%       3345 ± 12%  interrupts.CPU152.CAL:Function_call_interrupts
     74.00 ± 54%    +135.8%     174.50 ± 27%  interrupts.CPU152.RES:Rescheduling_interrupts
    431.00 ± 36%    +151.4%       1083 ± 31%  interrupts.CPU152.TLB:TLB_shootdowns
    580.25 ± 59%     +91.8%       1112 ± 26%  interrupts.CPU156.TLB:TLB_shootdowns
      8427 ±  4%     -53.3%       3932 ± 22%  interrupts.CPU16.NMI:Non-maskable_interrupts
      8427 ±  4%     -53.3%       3932 ± 22%  interrupts.CPU16.PMI:Performance_monitoring_interrupts
    234.75 ± 15%     -46.6%     125.25 ± 53%  interrupts.CPU16.RES:Rescheduling_interrupts
      7739 ±  9%     -48.9%       3953 ± 30%  interrupts.CPU164.NMI:Non-maskable_interrupts
      7739 ±  9%     -48.9%       3953 ± 30%  interrupts.CPU164.PMI:Performance_monitoring_interrupts
      3669 ±  7%     -16.7%       3055 ±  8%  interrupts.CPU165.CAL:Function_call_interrupts
      7853 ± 16%     -49.9%       3933 ± 47%  interrupts.CPU165.NMI:Non-maskable_interrupts
      7853 ± 16%     -49.9%       3933 ± 47%  interrupts.CPU165.PMI:Performance_monitoring_interrupts
      1430 ± 17%     -44.7%     790.50 ± 35%  interrupts.CPU165.TLB:TLB_shootdowns
      5312 ± 18%     +35.9%       7220 ± 18%  interrupts.CPU168.NMI:Non-maskable_interrupts
      5312 ± 18%     +35.9%       7220 ± 18%  interrupts.CPU168.PMI:Performance_monitoring_interrupts
      3547 ±  3%     -16.2%       2972 ± 11%  interrupts.CPU169.CAL:Function_call_interrupts
    202.25 ± 24%     -56.2%      88.50 ± 64%  interrupts.CPU169.RES:Rescheduling_interrupts
      8001 ± 13%     -45.4%       4370 ± 46%  interrupts.CPU17.NMI:Non-maskable_interrupts
      8001 ± 13%     -45.4%       4370 ± 46%  interrupts.CPU17.PMI:Performance_monitoring_interrupts
      8053 ±  8%     -42.5%       4627 ± 58%  interrupts.CPU172.NMI:Non-maskable_interrupts
      8053 ±  8%     -42.5%       4627 ± 58%  interrupts.CPU172.PMI:Performance_monitoring_interrupts
    159.75 ± 33%     -65.7%      54.75 ± 72%  interrupts.CPU173.RES:Rescheduling_interrupts
      8384           -53.1%       3930 ± 47%  interrupts.CPU176.NMI:Non-maskable_interrupts
      8384           -53.1%       3930 ± 47%  interrupts.CPU176.PMI:Performance_monitoring_interrupts
    636.00 ± 41%     +65.0%       1049 ± 35%  interrupts.CPU179.TLB:TLB_shootdowns
      3017 ± 12%     +15.3%       3479 ± 11%  interrupts.CPU189.CAL:Function_call_interrupts
    275.00 ±  6%     -38.6%     168.75 ± 22%  interrupts.CPU2.RES:Rescheduling_interrupts
      1540 ± 12%     -23.6%       1176 ± 13%  interrupts.CPU2.TLB:TLB_shootdowns
    260.00 ± 19%     -51.2%     127.00 ± 39%  interrupts.CPU20.RES:Rescheduling_interrupts
    253.75 ± 11%     -71.3%      72.75 ± 69%  interrupts.CPU27.RES:Rescheduling_interrupts
      1480 ± 11%     -60.9%     578.25 ± 81%  interrupts.CPU27.TLB:TLB_shootdowns
    219.00 ± 13%     -37.9%     136.00 ± 10%  interrupts.CPU3.RES:Rescheduling_interrupts
    714.50 ± 49%     +83.4%       1310 ± 22%  interrupts.CPU30.TLB:TLB_shootdowns
      3577 ±  6%     -14.7%       3053 ± 11%  interrupts.CPU35.CAL:Function_call_interrupts
    248.50 ± 10%     -50.8%     122.25 ± 61%  interrupts.CPU35.RES:Rescheduling_interrupts
      1340 ± 12%     -49.2%     681.50 ± 41%  interrupts.CPU35.TLB:TLB_shootdowns
    239.25 ± 12%     -62.7%      89.25 ± 95%  interrupts.CPU4.RES:Rescheduling_interrupts
    225.50 ± 20%     -24.3%     170.75 ± 27%  interrupts.CPU42.RES:Rescheduling_interrupts
    200.50 ± 31%     -52.2%      95.75 ± 44%  interrupts.CPU46.RES:Rescheduling_interrupts
    377.75 ± 65%    +179.2%       1054 ± 21%  interrupts.CPU49.TLB:TLB_shootdowns
    153.00 ± 17%     -42.3%      88.25 ± 25%  interrupts.CPU55.RES:Rescheduling_interrupts
    212.75 ± 14%     -67.5%      69.25 ± 37%  interrupts.CPU56.RES:Rescheduling_interrupts
      1383 ± 13%     -49.1%     703.75 ± 50%  interrupts.CPU56.TLB:TLB_shootdowns
    242.50 ± 17%     -57.6%     102.75 ±103%  interrupts.CPU57.RES:Rescheduling_interrupts
      3764 ±  9%     -20.9%       2976 ±  8%  interrupts.CPU60.CAL:Function_call_interrupts
    218.75 ± 24%     -61.7%      83.75 ± 52%  interrupts.CPU60.RES:Rescheduling_interrupts
      1316 ± 23%     -48.7%     675.25 ± 45%  interrupts.CPU60.TLB:TLB_shootdowns
    204.00 ±  8%     -60.8%      80.00 ± 66%  interrupts.CPU7.RES:Rescheduling_interrupts
    249.25 ± 12%     -26.4%     183.50 ± 23%  interrupts.CPU74.RES:Rescheduling_interrupts
    124.25 ± 31%     +46.1%     181.50 ± 21%  interrupts.CPU77.RES:Rescheduling_interrupts
      3508 ±  8%     -10.4%       3144 ± 12%  interrupts.CPU78.CAL:Function_call_interrupts
      6194 ± 35%     -42.3%       3574 ± 34%  interrupts.CPU8.NMI:Non-maskable_interrupts
      6194 ± 35%     -42.3%       3574 ± 34%  interrupts.CPU8.PMI:Performance_monitoring_interrupts
      5092 ± 25%     +67.3%       8522        interrupts.CPU80.NMI:Non-maskable_interrupts
      5092 ± 25%     +67.3%       8522        interrupts.CPU80.PMI:Performance_monitoring_interrupts
    169.25 ± 29%     -54.2%      77.50 ± 46%  interrupts.CPU90.RES:Rescheduling_interrupts
    216.00 ±  7%     -73.8%      56.50 ± 61%  interrupts.CPU93.RES:Rescheduling_interrupts
    254.50 ±  3%     -26.8%     186.25 ± 15%  interrupts.CPU96.RES:Rescheduling_interrupts
      1372 ± 12%     -16.3%       1149 ± 18%  interrupts.CPU96.TLB:TLB_shootdowns
     92.50 ± 23%     +98.1%     183.25 ± 18%  interrupts.CPU97.RES:Rescheduling_interrupts
    158.75 ± 98%    +221.1%     509.75 ± 34%  interrupts.CPU98.TLB:TLB_shootdowns
     28796 ±  3%     -17.4%      23785 ± 15%  interrupts.RES:Rescheduling_interrupts



***************************************************************************************************
lkp-csl-2ap2: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode:
  gcc-9/performance/x86_64-rhel-8.3/thread/16/debian-10.4-x86_64-20200603.cgz/lkp-csl-2ap2/futex4/will-it-scale/0x5003003

commit: 
  9720a64438 ("sched: Report local wake up on resched blind zone within idle loop")
  8e01c5f104 ("entry: Report local wake up on resched blind zone while resuming to user")

9720a64438d901da 8e01c5f10451c019e384d68ee8e 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
 1.068e+08            -1.5%  1.052e+08        will-it-scale.16.threads
   6674552            -1.5%    6571881        will-it-scale.per_thread_ops
 1.068e+08            -1.5%  1.052e+08        will-it-scale.workload
    540984 ± 27%     -45.1%     296787 ± 49%  numa-numastat.node2.local_node
      1158 ±  6%     -10.7%       1034 ±  7%  slabinfo.file_lock_cache.active_objs
      1158 ±  6%     -10.7%       1034 ±  7%  slabinfo.file_lock_cache.num_objs
      6900 ±127%     -95.6%     301.00 ±102%  softirqs.CPU11.NET_RX
     21404 ±  9%     +30.2%      27867 ±  6%  softirqs.CPU111.SCHED
     23371 ±  8%     -26.7%      17133 ±  8%  softirqs.CPU15.SCHED
    243.75 ± 63%    +112.1%     517.00 ± 16%  numa-vmstat.node0.nr_page_table_pages
     16717 ±  4%     +13.7%      19002 ±  3%  numa-vmstat.node0.nr_slab_unreclaimable
    425644 ± 14%     +16.4%     495424 ±  8%  numa-vmstat.node0.numa_local
      1374 ± 55%     -71.5%     391.25 ±114%  numa-vmstat.node1.nr_shmem
      4803 ± 17%     +60.0%       7686 ± 39%  numa-vmstat.node1.nr_slab_reclaimable
    775917 ±  8%     +10.3%     855691 ±  3%  numa-meminfo.node0.MemUsed
    977.75 ± 63%    +112.9%       2081 ± 15%  numa-meminfo.node0.PageTables
     66871 ±  4%     +13.7%      76009 ±  3%  numa-meminfo.node0.SUnreclaim
     19215 ± 17%     +60.0%      30749 ± 39%  numa-meminfo.node1.KReclaimable
     19215 ± 17%     +60.0%      30749 ± 39%  numa-meminfo.node1.SReclaimable
      5497 ± 55%     -71.5%       1566 ±114%  numa-meminfo.node1.Shmem
      0.01 ± 48%    +114.3%       0.01 ± 38%  perf-sched.sch_delay.avg.ms.schedule_timeout.wait_for_completion.__flush_work.lru_add_drain_all
      0.01 ± 15%    +373.7%       0.04 ±108%  perf-sched.sch_delay.max.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
      0.01 ±  8%    +205.4%       0.04 ±102%  perf-sched.sch_delay.max.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
      0.01 ± 48%    +114.3%       0.01 ± 38%  perf-sched.sch_delay.max.ms.schedule_timeout.wait_for_completion.__flush_work.lru_add_drain_all
      7256 ±  3%     +12.9%       8193 ±  3%  perf-sched.total_wait_and_delay.max.ms
      7256 ±  3%     +12.9%       8193 ±  3%  perf-sched.total_wait_time.max.ms
    595.05 ± 11%     -12.9%     518.40 ±  5%  perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
      5903 ± 21%     +38.8%       8193 ±  3%  perf-sched.wait_and_delay.max.ms.worker_thread.kthread.ret_from_fork
    595.04 ± 11%     -12.9%     518.39 ±  5%  perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
      5903 ± 21%     +38.8%       8193 ±  3%  perf-sched.wait_time.max.ms.worker_thread.kthread.ret_from_fork
      0.98 ±  6%      +0.1        1.12 ±  8%  perf-profile.calltrace.cycles-pp.syscall_enter_from_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.syscall
      0.91 ±  9%      +0.1        1.05 ±  7%  perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe.syscall
      0.30 ±  7%      +0.0        0.34 ±  6%  perf-profile.children.cycles-pp.scheduler_tick
      0.11 ± 12%      +0.1        0.16 ± 31%  perf-profile.children.cycles-pp.tick_irq_enter
      0.58 ± 10%      +0.1        0.68 ± 11%  perf-profile.children.cycles-pp.tick_sched_timer
      0.00            +0.1        0.15 ± 14%  perf-profile.children.cycles-pp.sched_resched_local_allow
      0.98 ±  6%      +0.1        1.13 ±  8%  perf-profile.children.cycles-pp.syscall_enter_from_user_mode
      1.16 ±  9%      +0.2        1.33 ±  8%  perf-profile.children.cycles-pp.exit_to_user_mode_prepare
      1.38 ±  8%      +0.2        1.56 ±  9%  perf-profile.children.cycles-pp.hrtimer_interrupt
      0.00            +0.3        0.28 ± 11%  perf-profile.children.cycles-pp.sched_resched_local_forbid
      0.00            +0.1        0.14 ±  9%  perf-profile.self.cycles-pp.sched_resched_local_forbid
      0.00            +0.1        0.14 ± 12%  perf-profile.self.cycles-pp.sched_resched_local_allow
 9.413e+09            +2.9%  9.684e+09        perf-stat.i.branch-instructions
 1.521e+10            +1.5%  1.544e+10        perf-stat.i.dTLB-loads
 1.174e+10            +2.1%  1.198e+10        perf-stat.i.dTLB-stores
  54241332            -1.8%   53267348        perf-stat.i.iTLB-load-misses
      1082            +2.1%       1104        perf-stat.i.instructions-per-iTLB-miss
    189.49            +2.1%     193.48        perf-stat.i.metric.M/sec
      1078            +2.1%       1100        perf-stat.overall.instructions-per-iTLB-miss
    165026            +1.8%     167961        perf-stat.overall.path-length
 9.381e+09            +2.9%  9.651e+09        perf-stat.ps.branch-instructions
 1.516e+10            +1.5%  1.538e+10        perf-stat.ps.dTLB-loads
  1.17e+10            +2.1%  1.194e+10        perf-stat.ps.dTLB-stores
  54057433            -1.8%   53087204        perf-stat.ps.iTLB-load-misses
     13120 ±126%     -95.7%     563.75 ±106%  interrupts.32:PCI-MSI.524290-edge.eth0-TxRx-1
      6947 ± 12%     -37.7%       4326 ± 34%  interrupts.CPU0.NMI:Non-maskable_interrupts
      6947 ± 12%     -37.7%       4326 ± 34%  interrupts.CPU0.PMI:Performance_monitoring_interrupts
     13120 ±126%     -95.7%     563.75 ±106%  interrupts.CPU11.32:PCI-MSI.524290-edge.eth0-TxRx-1
    288.50 ± 18%     -40.6%     171.25 ± 21%  interrupts.CPU111.TLB:TLB_shootdowns
    101.25 ± 28%     +49.1%     151.00 ± 19%  interrupts.CPU122.NMI:Non-maskable_interrupts
    101.25 ± 28%     +49.1%     151.00 ± 19%  interrupts.CPU122.PMI:Performance_monitoring_interrupts
    118.50 ±  5%     +15.4%     136.75 ±  8%  interrupts.CPU123.NMI:Non-maskable_interrupts
    118.50 ±  5%     +15.4%     136.75 ±  8%  interrupts.CPU123.PMI:Performance_monitoring_interrupts
     99.25 ± 24%     +38.5%     137.50 ± 11%  interrupts.CPU125.NMI:Non-maskable_interrupts
     99.25 ± 24%     +38.5%     137.50 ± 11%  interrupts.CPU125.PMI:Performance_monitoring_interrupts
     98.25 ± 23%     +45.0%     142.50 ± 24%  interrupts.CPU126.NMI:Non-maskable_interrupts
     98.25 ± 23%     +45.0%     142.50 ± 24%  interrupts.CPU126.PMI:Performance_monitoring_interrupts
    114.25 ±  5%     +24.1%     141.75 ± 12%  interrupts.CPU135.NMI:Non-maskable_interrupts
    114.25 ±  5%     +24.1%     141.75 ± 12%  interrupts.CPU135.PMI:Performance_monitoring_interrupts
     99.00 ± 23%     +33.6%     132.25 ± 12%  interrupts.CPU137.NMI:Non-maskable_interrupts
     99.00 ± 23%     +33.6%     132.25 ± 12%  interrupts.CPU137.PMI:Performance_monitoring_interrupts
     98.75 ± 24%     +31.9%     130.25 ± 13%  interrupts.CPU138.NMI:Non-maskable_interrupts
     98.75 ± 24%     +31.9%     130.25 ± 13%  interrupts.CPU138.PMI:Performance_monitoring_interrupts
     98.50 ± 24%     +31.0%     129.00 ± 12%  interrupts.CPU139.NMI:Non-maskable_interrupts
     98.50 ± 24%     +31.0%     129.00 ± 12%  interrupts.CPU139.PMI:Performance_monitoring_interrupts
     98.25 ± 24%     +32.8%     130.50 ± 12%  interrupts.CPU140.NMI:Non-maskable_interrupts
     98.25 ± 24%     +32.8%     130.50 ± 12%  interrupts.CPU140.PMI:Performance_monitoring_interrupts
     84.00 ± 30%     +55.1%     130.25 ± 11%  interrupts.CPU141.NMI:Non-maskable_interrupts
     84.00 ± 30%     +55.1%     130.25 ± 11%  interrupts.CPU141.PMI:Performance_monitoring_interrupts
     86.50 ± 26%     +51.7%     131.25 ± 12%  interrupts.CPU142.NMI:Non-maskable_interrupts
     86.50 ± 26%     +51.7%     131.25 ± 12%  interrupts.CPU142.PMI:Performance_monitoring_interrupts
     84.00 ± 30%     +97.3%     165.75 ± 25%  interrupts.CPU143.NMI:Non-maskable_interrupts
     84.00 ± 30%     +97.3%     165.75 ± 25%  interrupts.CPU143.PMI:Performance_monitoring_interrupts
    253.50 ± 20%     +43.8%     364.50 ± 14%  interrupts.CPU15.TLB:TLB_shootdowns
    101.50 ± 24%     +32.3%     134.25 ± 12%  interrupts.CPU150.NMI:Non-maskable_interrupts
    101.50 ± 24%     +32.3%     134.25 ± 12%  interrupts.CPU150.PMI:Performance_monitoring_interrupts
    121.75 ± 10%    +115.4%     262.25 ± 84%  interrupts.CPU153.NMI:Non-maskable_interrupts
    121.75 ± 10%    +115.4%     262.25 ± 84%  interrupts.CPU153.PMI:Performance_monitoring_interrupts
     77.75 ± 40%     +71.1%     133.00 ± 12%  interrupts.CPU167.NMI:Non-maskable_interrupts
     77.75 ± 40%     +71.1%     133.00 ± 12%  interrupts.CPU167.PMI:Performance_monitoring_interrupts
     77.75 ± 30%    +137.3%     184.50 ± 49%  interrupts.CPU169.NMI:Non-maskable_interrupts
     77.75 ± 30%    +137.3%     184.50 ± 49%  interrupts.CPU169.PMI:Performance_monitoring_interrupts
      7583 ± 14%     -46.7%       4043 ± 31%  interrupts.CPU2.NMI:Non-maskable_interrupts
      7583 ± 14%     -46.7%       4043 ± 31%  interrupts.CPU2.PMI:Performance_monitoring_interrupts
     85.25 ± 33%     +96.8%     167.75 ± 31%  interrupts.CPU26.NMI:Non-maskable_interrupts
     85.25 ± 33%     +96.8%     167.75 ± 31%  interrupts.CPU26.PMI:Performance_monitoring_interrupts
    100.50 ± 27%     +46.5%     147.25 ± 17%  interrupts.CPU29.NMI:Non-maskable_interrupts
    100.50 ± 27%     +46.5%     147.25 ± 17%  interrupts.CPU29.PMI:Performance_monitoring_interrupts
    115.00 ±  7%     +16.7%     134.25 ± 12%  interrupts.CPU37.NMI:Non-maskable_interrupts
    115.00 ±  7%     +16.7%     134.25 ± 12%  interrupts.CPU37.PMI:Performance_monitoring_interrupts
    113.75 ±  4%     +16.5%     132.50 ± 11%  interrupts.CPU38.NMI:Non-maskable_interrupts
    113.75 ±  4%     +16.5%     132.50 ± 11%  interrupts.CPU38.PMI:Performance_monitoring_interrupts
    113.50 ±  4%     +22.0%     138.50 ± 10%  interrupts.CPU39.NMI:Non-maskable_interrupts
    113.50 ±  4%     +22.0%     138.50 ± 10%  interrupts.CPU39.PMI:Performance_monitoring_interrupts
    113.25 ±  5%     +16.1%     131.50 ± 11%  interrupts.CPU41.NMI:Non-maskable_interrupts
    113.25 ±  5%     +16.1%     131.50 ± 11%  interrupts.CPU41.PMI:Performance_monitoring_interrupts
    101.50 ± 20%     +28.1%     130.00 ± 12%  interrupts.CPU46.NMI:Non-maskable_interrupts
    101.50 ± 20%     +28.1%     130.00 ± 12%  interrupts.CPU46.PMI:Performance_monitoring_interrupts
     99.25 ± 24%     +50.6%     149.50 ± 13%  interrupts.CPU47.NMI:Non-maskable_interrupts
     99.25 ± 24%     +50.6%     149.50 ± 13%  interrupts.CPU47.PMI:Performance_monitoring_interrupts
     87.25 ± 30%    +168.2%     234.00 ± 71%  interrupts.CPU57.NMI:Non-maskable_interrupts
     87.25 ± 30%    +168.2%     234.00 ± 71%  interrupts.CPU57.PMI:Performance_monitoring_interrupts
     91.50 ± 26%     +57.1%     143.75 ± 24%  interrupts.CPU58.NMI:Non-maskable_interrupts
     91.50 ± 26%     +57.1%     143.75 ± 24%  interrupts.CPU58.PMI:Performance_monitoring_interrupts
     36.25 ±103%     -77.9%       8.00 ± 63%  interrupts.CPU6.RES:Rescheduling_interrupts
      7788 ± 14%     -37.3%       4883 ± 25%  interrupts.CPU7.NMI:Non-maskable_interrupts
      7788 ± 14%     -37.3%       4883 ± 25%  interrupts.CPU7.PMI:Performance_monitoring_interrupts
    251.75 ± 30%     -33.1%     168.50 ± 20%  interrupts.CPU97.TLB:TLB_shootdowns





Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


Thanks,
Oliver Sang


View attachment "config-5.11.0-rc2-00007-g8e01c5f10451" of type "text/plain" (172412 bytes)

View attachment "job-script" of type "text/plain" (8004 bytes)

View attachment "job.yaml" of type "text/plain" (5357 bytes)

View attachment "reproduce" of type "text/plain" (279 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ