lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20220518092414.GA15472@xsang-OptiPlex-9020>
Date:   Wed, 18 May 2022 17:24:14 +0800
From:   kernel test robot <oliver.sang@...el.com>
To:     Mel Gorman <mgorman@...hsingularity.net>
Cc:     0day robot <lkp@...el.com>, LKML <linux-kernel@...r.kernel.org>,
        lkp@...ts.01.org, ying.huang@...el.com, feng.tang@...el.com,
        zhengjun.xing@...ux.intel.com, fengwei.yin@...el.com,
        Peter Zijlstra <peterz@...radead.org>,
        Ingo Molnar <mingo@...nel.org>,
        Vincent Guittot <vincent.guittot@...aro.org>,
        Valentin Schneider <valentin.schneider@....com>,
        Aubrey Li <aubrey.li@...ux.intel.com>,
        Mel Gorman <mgorman@...hsingularity.net>, yu.c.chen@...el.com
Subject: [sched/numa]  bb2dee337b:  unixbench.score -11.2% regression



Greeting,

FYI, we noticed a -11.2% regression of unixbench.score due to commit:


commit: bb2dee337bd7d314eb7c7627e1afd754f86566bc ("[PATCH 3/4] sched/numa: Apply imbalance limitations consistently")
url: https://github.com/intel-lab-lkp/linux/commits/Mel-Gorman/Mitigate-inconsistent-NUMA-imbalance-behaviour/20220511-223233
base: https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git d70522fc541224b8351ac26f4765f2c6268f8d72
patch link: https://lore.kernel.org/lkml/20220511143038.4620-4-mgorman@techsingularity.net

in testcase: unixbench
on test machine: 128 threads 2 sockets Intel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz with 256G memory
with following parameters:

	runtime: 300s
	nr_task: 1
	test: shell8
	cpufreq_governor: performance
	ucode: 0xd000331

test-description: UnixBench is the original BYTE UNIX benchmark suite aims to test performance of Unix-like system.
test-url: https://github.com/kdlucas/byte-unixbench

In addition to that, the commit also has significant impact on the following tests:

+------------------+-------------------------------------------------------------------------------------+
| testcase: change | fsmark: fsmark.files_per_sec -21.5% regression                                      |
| test machine     | 192 threads 4 sockets Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory |
| test parameters  | cpufreq_governor=performance                                                        |
|                  | disk=1SSD                                                                           |
|                  | filesize=8K                                                                         |
|                  | fs=f2fs                                                                             |
|                  | iterations=8                                                                        |
|                  | nr_directories=16d                                                                  |
|                  | nr_files_per_directory=256fpd                                                       |
|                  | nr_threads=4                                                                        |
|                  | sync_method=fsyncBeforeClose                                                        |
|                  | test_size=72G                                                                       |
|                  | ucode=0x500320a                                                                     |
+------------------+-------------------------------------------------------------------------------------+


If you fix the issue, kindly add following tag
Reported-by: kernel test robot <oliver.sang@...el.com>


Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        sudo bin/lkp install job.yaml           # job file is attached in this email
        bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run
        sudo bin/lkp run generated-yaml-file

        # if come across any failure that blocks the test,
        # please remove ~/.lkp and /lkp dir to run from a clean state.

=========================================================================================
compiler/cpufreq_governor/kconfig/nr_task/rootfs/runtime/tbox_group/test/testcase/ucode:
  gcc-11/performance/x86_64-rhel-8.3/1/debian-10.4-x86_64-20200603.cgz/300s/lkp-icl-2sp2/shell8/unixbench/0xd000331

commit: 
  19f9b71e9f ("sched/numa: Do not swap tasks between nodes when spare capacity is available")
  bb2dee337b ("sched/numa: Apply imbalance limitations consistently")

19f9b71e9fb9c42f bb2dee337bd7d314eb7c7627e1a 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
      9456           -11.2%       8400        unixbench.score
     47643 ±  2%      -9.4%      43144 ±  2%  unixbench.time.involuntary_context_switches
      1595 ±  6%    +138.4%       3802 ±  5%  unixbench.time.major_page_faults
  45950495           -11.1%   40848424        unixbench.time.minor_page_faults
    173.45           +18.9%     206.31        unixbench.time.system_time
    190.25           -16.8%     158.20        unixbench.time.user_time
   1306482           -12.6%    1141414        unixbench.time.voluntary_context_switches
    357454           -11.2%     317544        unixbench.workload
     31.47 ±  2%      +5.2%      33.10        turbostat.RAMWatt
      0.14 ±  4%      +0.0        0.16        mpstat.cpu.all.soft%
      1.61            -0.2        1.44        mpstat.cpu.all.usr%
      1149 ± 10%     +32.8%       1527 ± 23%  sched_debug.cfs_rq:/.runnable_avg.max
      1149 ± 10%     +32.8%       1527 ± 23%  sched_debug.cfs_rq:/.util_avg.max
     53195           -10.5%      47596        vmstat.system.cs
    271972            -6.1%     255312 ± 12%  vmstat.system.in
  13345450 ± 12%     -26.6%    9796608 ±  5%  meminfo.DirectMap2M
     22508 ±  2%     +13.0%      25424 ±  3%  meminfo.KernelStack
      7966 ±  3%    +151.2%      20013 ± 15%  meminfo.PageTables
     56391 ± 18%     -57.7%      23845 ± 17%  numa-vmstat.node0.nr_anon_pages
     58577 ± 17%     -55.3%      26191 ± 17%  numa-vmstat.node0.nr_inactive_anon
     11721 ±  7%     +18.0%      13830 ±  5%  numa-vmstat.node0.nr_kernel_stack
      1110 ± 34%    +126.3%       2512 ± 27%  numa-vmstat.node0.nr_page_table_pages
     58577 ± 17%     -55.3%      26192 ± 17%  numa-vmstat.node0.nr_zone_inactive_anon
     16891 ± 63%    +205.2%      51557 ±  6%  numa-vmstat.node1.nr_anon_pages
     18994 ± 53%    +183.0%      53759 ±  6%  numa-vmstat.node1.nr_inactive_anon
    894.83 ± 41%    +182.2%       2525 ± 16%  numa-vmstat.node1.nr_page_table_pages
     18994 ± 53%    +183.0%      53759 ±  6%  numa-vmstat.node1.nr_zone_inactive_anon
     83590 ± 13%     -73.7%      21988 ± 32%  numa-meminfo.node0.AnonHugePages
    225657 ± 18%     -58.0%      94847 ± 18%  numa-meminfo.node0.AnonPages
    231652 ± 17%     -55.3%     103657 ± 16%  numa-meminfo.node0.AnonPages.max
    234525 ± 17%     -55.5%     104341 ± 18%  numa-meminfo.node0.Inactive
    234397 ± 17%     -55.5%     104267 ± 18%  numa-meminfo.node0.Inactive(anon)
     11724 ±  7%     +17.5%      13781 ±  5%  numa-meminfo.node0.KernelStack
      4472 ± 34%    +117.1%       9708 ± 31%  numa-meminfo.node0.PageTables
     15239 ± 75%    +401.2%      76387 ± 10%  numa-meminfo.node1.AnonHugePages
     67256 ± 63%    +206.3%     205994 ±  6%  numa-meminfo.node1.AnonPages
     73568 ± 58%    +193.1%     215644 ±  6%  numa-meminfo.node1.AnonPages.max
     75737 ± 53%    +183.9%     215053 ±  6%  numa-meminfo.node1.Inactive
     75709 ± 53%    +183.9%     214971 ±  6%  numa-meminfo.node1.Inactive(anon)
      3559 ± 42%    +187.1%      10216 ±  8%  numa-meminfo.node1.PageTables
     73240            +2.7%      75223        proc-vmstat.nr_anon_pages
     77537            +2.9%      79817        proc-vmstat.nr_inactive_anon
     22505 ±  2%     +12.8%      25387 ±  3%  proc-vmstat.nr_kernel_stack
      2003 ±  3%    +148.7%       4982 ± 20%  proc-vmstat.nr_page_table_pages
     61769            -1.5%      60836        proc-vmstat.nr_slab_unreclaimable
     77537            +2.9%      79817        proc-vmstat.nr_zone_inactive_anon
  33311917           -11.1%   29603705        proc-vmstat.numa_hit
  33210979           -11.2%   29488489        proc-vmstat.numa_local
      2797 ±  6%    +459.4%      15647        proc-vmstat.pgactivate
  33306223           -11.1%   29598069        proc-vmstat.pgalloc_normal
  46257005           -11.0%   41162286        proc-vmstat.pgfault
  33119124           -11.2%   29411010        proc-vmstat.pgfree
   2595965           -11.4%    2300851        proc-vmstat.pgreuse
      1506            -9.9%       1357        proc-vmstat.thp_fault_alloc
    635705           -11.2%     564736        proc-vmstat.unevictable_pgs_culled
     10.72            +3.8%      11.12        perf-stat.i.MPKI
 4.952e+09           -10.1%  4.454e+09        perf-stat.i.branch-instructions
      1.77            +0.0        1.80        perf-stat.i.branch-miss-rate%
  86802815            -8.5%   79414679        perf-stat.i.branch-misses
      4.38           +10.3       14.65        perf-stat.i.cache-miss-rate%
  10798607          +238.6%   36565778        perf-stat.i.cache-misses
 2.638e+08            -6.7%  2.461e+08        perf-stat.i.cache-references
     54996           -10.7%      49115        perf-stat.i.context-switches
      0.85           +10.2%       0.93 ±  3%  perf-stat.i.cpi
      1071 ±  3%     +91.7%       2053        perf-stat.i.cpu-migrations
      1995 ±  2%     -62.9%     739.38 ±  4%  perf-stat.i.cycles-between-cache-misses
   2713827 ±  2%      -6.7%    2531773        perf-stat.i.dTLB-load-misses
 6.269e+09           -10.4%  5.616e+09        perf-stat.i.dTLB-loads
   3635648           -11.1%    3230371        perf-stat.i.dTLB-store-misses
  3.66e+09           -11.0%  3.256e+09        perf-stat.i.dTLB-stores
 2.398e+10           -10.0%  2.158e+10        perf-stat.i.instructions
      1.19            -9.4%       1.08 ±  3%  perf-stat.i.ipc
     25.18 ±  5%    +137.6%      59.84 ±  4%  perf-stat.i.major-faults
    102.15           +77.7%     181.53        perf-stat.i.metric.K/sec
    118.29           -10.4%     106.01        perf-stat.i.metric.M/sec
    709279           -11.0%     631605        perf-stat.i.minor-faults
     67.04           +25.5       92.50        perf-stat.i.node-load-miss-rate%
    820091          +792.0%    7315370        perf-stat.i.node-load-misses
    406593 ±  4%     +30.7%     531337        perf-stat.i.node-loads
      5.29 ±  4%     +34.5       39.84        perf-stat.i.node-store-miss-rate%
    190224 ±  3%   +1726.4%    3474284        perf-stat.i.node-store-misses
   4299275           +19.9%    5154130        perf-stat.i.node-stores
    709304           -10.9%     631665        perf-stat.i.page-faults
     11.00            +3.7%      11.41        perf-stat.overall.MPKI
      1.75            +0.0        1.78        perf-stat.overall.branch-miss-rate%
      4.09           +10.8       14.86        perf-stat.overall.cache-miss-rate%
      0.84           +10.8%       0.93 ±  2%  perf-stat.overall.cpi
      1855           -70.5%     546.43 ±  2%  perf-stat.overall.cycles-between-cache-misses
      1.20            -9.7%       1.08 ±  3%  perf-stat.overall.ipc
     66.87           +26.4       93.23        perf-stat.overall.node-load-miss-rate%
      4.24 ±  4%     +36.0       40.26        perf-stat.overall.node-store-miss-rate%
   4254574            +1.1%    4300586        perf-stat.overall.path-length
 4.874e+09           -10.0%  4.384e+09        perf-stat.ps.branch-instructions
  85435301            -8.5%   78170086        perf-stat.ps.branch-misses
  10628137          +238.7%   35992941        perf-stat.ps.cache-misses
 2.597e+08            -6.7%  2.423e+08        perf-stat.ps.cache-references
     54129           -10.7%      48346        perf-stat.ps.context-switches
      1054 ±  3%     +91.7%       2021        perf-stat.ps.cpu-migrations
   2670954 ±  2%      -6.7%    2492108        perf-stat.ps.dTLB-load-misses
  6.17e+09           -10.4%  5.528e+09        perf-stat.ps.dTLB-loads
   3578206           -11.1%    3179761        perf-stat.ps.dTLB-store-misses
 3.602e+09           -11.0%  3.205e+09        perf-stat.ps.dTLB-stores
  2.36e+10           -10.0%  2.124e+10        perf-stat.ps.instructions
     24.78 ±  5%    +137.7%      58.90 ±  4%  perf-stat.ps.major-faults
    698073           -10.9%     621710        perf-stat.ps.minor-faults
    807155          +792.1%    7200767        perf-stat.ps.node-load-misses
    400170 ±  4%     +30.7%     523004        perf-stat.ps.node-loads
    187229 ±  4%   +1726.6%    3419862        perf-stat.ps.node-store-misses
   4231370           +19.9%    5073383        perf-stat.ps.node-stores
    698098           -10.9%     621769        perf-stat.ps.page-faults
 1.521e+12           -10.2%  1.366e+12        perf-stat.total.instructions
     13.69 ± 30%     -10.9        2.76 ± 69%  perf-profile.calltrace.cycles-pp.asm_sysvec_call_function.acpi_idle_do_entry.acpi_idle_enter.cpuidle_enter_state.cpuidle_enter
     14.60 ± 48%     -10.2        4.38 ± 36%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_write
     14.60 ± 48%     -10.2        4.38 ± 36%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
     14.60 ± 48%     -10.2        4.38 ± 36%  perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
     14.60 ± 48%     -10.2        4.38 ± 36%  perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
     14.60 ± 48%     -10.2        4.38 ± 36%  perf-profile.calltrace.cycles-pp.__libc_write
     14.60 ± 48%     -10.2        4.38 ± 36%  perf-profile.calltrace.cycles-pp.generic_file_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64
     14.60 ± 48%     -10.2        4.38 ± 36%  perf-profile.calltrace.cycles-pp.__generic_file_write_iter.generic_file_write_iter.new_sync_write.vfs_write.ksys_write
     14.53 ± 48%     -10.1        4.38 ± 36%  perf-profile.calltrace.cycles-pp.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.new_sync_write.vfs_write
      6.10 ± 30%      -4.9        1.22 ± 65%  perf-profile.calltrace.cycles-pp.event_function_call.perf_event_release_kernel.perf_release.__fput.task_work_run
      5.87 ± 31%      -4.7        1.22 ± 65%  perf-profile.calltrace.cycles-pp.smp_call_function_single.event_function_call.perf_event_release_kernel.perf_release.__fput
      5.40 ± 33%      -4.1        1.26 ± 81%  perf-profile.calltrace.cycles-pp.asm_exc_page_fault.fault_in_readable.fault_in_iov_iter_readable.generic_perform_write.__generic_file_write_iter
      5.62 ± 53%      -4.0        1.67 ± 54%  perf-profile.calltrace.cycles-pp.fault_in_iov_iter_readable.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.new_sync_write
      3.54 ± 50%      -2.1        1.42 ± 39%  perf-profile.calltrace.cycles-pp.copy_page_from_iter_atomic.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.new_sync_write
      0.56 ± 75%      +0.6        1.19 ± 25%  perf-profile.calltrace.cycles-pp.unmap_vmas.exit_mmap.mmput.exit_mm.do_exit
      0.51 ± 77%      +0.7        1.19 ± 25%  perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.exit_mmap.mmput.exit_mm
      0.28 ±143%      +0.7        1.01 ± 23%  perf-profile.calltrace.cycles-pp.fast_imageblit.sys_imageblit.drm_fbdev_fb_imageblit.bit_putcs.fbcon_putcs
      0.28 ±143%      +0.7        1.01 ± 23%  perf-profile.calltrace.cycles-pp.drm_fbdev_fb_imageblit.bit_putcs.fbcon_putcs.fbcon_redraw.fbcon_scroll
      0.28 ±143%      +0.7        1.01 ± 23%  perf-profile.calltrace.cycles-pp.sys_imageblit.drm_fbdev_fb_imageblit.bit_putcs.fbcon_putcs.fbcon_redraw
      0.40 ±106%      +0.9        1.32 ± 52%  perf-profile.calltrace.cycles-pp.zap_pte_range.zap_pmd_range.unmap_page_range.unmap_vmas.exit_mmap
      0.40 ±106%      +1.2        1.56 ± 42%  perf-profile.calltrace.cycles-pp.zap_pmd_range.unmap_page_range.unmap_vmas.exit_mmap.mmput
      1.50 ±113%      +1.9        3.45 ± 33%  perf-profile.calltrace.cycles-pp.delay_tsc.wait_for_xmitr.serial8250_console_putchar.uart_console_write.serial8250_console_write
      1.93 ± 26%      +2.4        4.29 ± 46%  perf-profile.calltrace.cycles-pp.bprm_execve.do_execveat_common.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe
      2.85 ± 30%      +2.4        5.21 ± 42%  perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.acpi_idle_do_entry.acpi_idle_enter
      2.66 ± 28%      +2.4        5.08 ± 44%  perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.acpi_idle_do_entry
      4.98 ± 32%      +3.5        8.50 ± 30%  perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.acpi_idle_do_entry.acpi_idle_enter.cpuidle_enter_state
      6.91 ± 24%      +3.7       10.62 ± 28%  perf-profile.calltrace.cycles-pp.acpi_idle_do_entry.acpi_idle_enter.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call
     17.43 ± 25%     +12.0       29.40 ± 24%  perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.acpi_idle_do_entry.acpi_idle_enter.cpuidle_enter_state.cpuidle_enter
     14.65 ± 48%     -10.3        4.38 ± 36%  perf-profile.children.cycles-pp.generic_file_write_iter
     14.65 ± 48%     -10.3        4.38 ± 36%  perf-profile.children.cycles-pp.__generic_file_write_iter
     14.61 ± 48%     -10.2        4.38 ± 36%  perf-profile.children.cycles-pp.generic_perform_write
     14.60 ± 48%     -10.2        4.38 ± 36%  perf-profile.children.cycles-pp.__libc_write
      7.23 ± 30%      -5.7        1.52 ± 66%  perf-profile.children.cycles-pp.asm_sysvec_call_function
      6.17 ± 29%      -4.9        1.24 ± 67%  perf-profile.children.cycles-pp.event_function_call
      6.05 ± 31%      -4.8        1.24 ± 67%  perf-profile.children.cycles-pp.smp_call_function_single
      5.62 ± 53%      -3.9        1.74 ± 45%  perf-profile.children.cycles-pp.fault_in_iov_iter_readable
      5.60 ± 53%      -3.9        1.74 ± 45%  perf-profile.children.cycles-pp.fault_in_readable
      3.54 ± 50%      -2.1        1.42 ± 39%  perf-profile.children.cycles-pp.copy_page_from_iter_atomic
      2.17 ± 53%      -1.7        0.46 ± 68%  perf-profile.children.cycles-pp.mutex_lock
      1.91 ± 54%      -1.6        0.36 ± 62%  perf-profile.children.cycles-pp.swevent_hlist_put_cpu
      2.11 ± 15%      -1.1        0.98 ± 46%  perf-profile.children.cycles-pp.__alloc_pages
      1.40 ± 42%      -1.1        0.32 ±111%  perf-profile.children.cycles-pp.shmem_write_end
      1.26 ± 30%      -1.0        0.22 ± 79%  perf-profile.children.cycles-pp.alloc_pages_vma
      1.34 ± 20%      -1.0        0.32 ± 78%  perf-profile.children.cycles-pp.get_page_from_freelist
      1.53 ± 47%      -0.9        0.64 ± 48%  perf-profile.children.cycles-pp.__might_resched
      0.72 ± 44%      -0.5        0.23 ± 86%  perf-profile.children.cycles-pp.__pagevec_lru_add
      0.61 ± 40%      -0.5        0.14 ±111%  perf-profile.children.cycles-pp.__pagevec_lru_add_fn
      0.46 ± 28%      -0.3        0.16 ± 82%  perf-profile.children.cycles-pp.call_rcu
      0.53 ± 40%      -0.2        0.30 ± 70%  perf-profile.children.cycles-pp.folio_add_lru
      0.03 ±223%      +0.2        0.23 ± 22%  perf-profile.children.cycles-pp.alloc_bprm
      0.03 ±223%      +0.3        0.37 ± 73%  perf-profile.children.cycles-pp.tick_sched_do_timer
      0.10 ±141%      +0.4        0.48 ± 50%  perf-profile.children.cycles-pp.unlink_file_vma
      0.30 ± 87%      +0.7        0.96 ± 43%  perf-profile.children.cycles-pp.__open64_nocancel
      1.03 ± 38%      +0.8        1.84 ± 35%  perf-profile.children.cycles-pp.zap_pmd_range
      1.08 ± 37%      +0.8        1.92 ± 39%  perf-profile.children.cycles-pp.unmap_page_range
      1.16 ± 41%      +0.9        2.03 ± 38%  perf-profile.children.cycles-pp.unmap_vmas
      0.89 ± 47%      +1.1        2.02 ± 33%  perf-profile.children.cycles-pp.exec_mmap
      0.92 ± 45%      +1.2        2.16 ± 33%  perf-profile.children.cycles-pp.begin_new_exec
      1.20 ± 51%      +1.4        2.62 ± 60%  perf-profile.children.cycles-pp.ktime_get
      1.70 ± 89%      +1.8        3.54 ± 33%  perf-profile.children.cycles-pp.delay_tsc
      1.83 ± 29%      +2.0        3.80 ± 39%  perf-profile.children.cycles-pp.exec_binprm
      1.83 ± 29%      +2.0        3.80 ± 39%  perf-profile.children.cycles-pp.search_binary_handler
      1.73 ± 26%      +2.0        3.72 ± 38%  perf-profile.children.cycles-pp.load_elf_binary
      3.12 ± 22%      +2.3        5.46 ± 47%  perf-profile.children.cycles-pp.hrtimer_interrupt
      1.96 ± 25%      +2.5        4.42 ± 43%  perf-profile.children.cycles-pp.bprm_execve
      2.57 ± 20%      +2.8        5.39 ± 45%  perf-profile.children.cycles-pp.__x64_sys_execve
      2.57 ± 20%      +2.8        5.39 ± 45%  perf-profile.children.cycles-pp.do_execveat_common
      5.50 ± 28%      +3.4        8.95 ± 32%  perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
     12.38 ± 24%      +7.3       19.71 ± 24%  perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
      5.84 ± 31%      -4.6        1.22 ± 65%  perf-profile.self.cycles-pp.smp_call_function_single
      2.92 ± 50%      -2.0        0.88 ± 53%  perf-profile.self.cycles-pp.fault_in_readable
      1.56 ± 58%      -1.2        0.33 ± 70%  perf-profile.self.cycles-pp.mutex_lock
      1.27 ± 54%      -1.0        0.23 ±114%  perf-profile.self.cycles-pp.shmem_write_end
      0.41 ± 32%      +0.3        0.71 ± 35%  perf-profile.self.cycles-pp.update_rq_clock
      0.03 ±223%      +0.3        0.34 ± 75%  perf-profile.self.cycles-pp.tick_sched_do_timer
      0.43 ± 48%      +0.6        1.00 ± 39%  perf-profile.self.cycles-pp.zap_pte_range
      0.99 ± 55%      +1.4        2.39 ± 70%  perf-profile.self.cycles-pp.ktime_get
      1.70 ± 89%      +1.8        3.54 ± 33%  perf-profile.self.cycles-pp.delay_tsc


***************************************************************************************************
lkp-csl-2ap4: 192 threads 4 sockets Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
=========================================================================================
compiler/cpufreq_governor/disk/filesize/fs/iterations/kconfig/nr_directories/nr_files_per_directory/nr_threads/rootfs/sync_method/tbox_group/test_size/testcase/ucode:
  gcc-11/performance/1SSD/8K/f2fs/8/x86_64-rhel-8.3/16d/256fpd/4/debian-10.4-x86_64-20200603.cgz/fsyncBeforeClose/lkp-csl-2ap4/72G/fsmark/0x500320a

commit: 
  19f9b71e9f ("sched/numa: Do not swap tasks between nodes when spare capacity is available")
  bb2dee337b ("sched/numa: Apply imbalance limitations consistently")

19f9b71e9fb9c42f bb2dee337bd7d314eb7c7627e1a 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
   4147579            +4.2%    4322142 ±  2%  fsmark.app_overhead
     38252 ±  4%     -21.5%      30012 ±  2%  fsmark.files_per_sec
    309.12 ±  3%     +22.6%     378.94 ±  2%  fsmark.time.elapsed_time
    309.12 ±  3%     +22.6%     378.94 ±  2%  fsmark.time.elapsed_time.max
     34264 ± 19%     -57.4%      14599 ± 13%  fsmark.time.involuntary_context_switches
    212677 ± 12%     -85.7%      30395 ± 42%  fsmark.time.minor_page_faults
    213.00 ±  2%     +10.8%     236.00        fsmark.time.percent_of_cpu_this_job_got
    628.50 ±  7%     +37.8%     866.05 ±  3%  fsmark.time.system_time
 5.771e+10 ±  3%     +22.7%  7.082e+10 ±  2%  cpuidle..time
 1.479e+08 ± 12%     +19.2%  1.764e+08        cpuidle..usage
      0.16 ±  3%      +0.0        0.21 ±  3%  mpstat.cpu.all.iowait%
      1.11 ±  4%      +0.1        1.21        mpstat.cpu.all.sys%
   1040563 ± 99%    +598.4%    7267234 ± 15%  numa-numastat.node0.local_node
   1121094 ± 92%    +554.2%    7334262 ± 15%  numa-numastat.node0.numa_hit
    361.90 ±  3%     +19.4%     432.10        uptime.boot
     66590 ±  3%     +19.8%      79803        uptime.idle
     28220 ± 56%     -68.3%       8945 ± 79%  numa-meminfo.node0.Shmem
    917.83 ±  8%     +26.1%       1157 ± 16%  numa-meminfo.node2.PageTables
    274322 ± 26%     -41.6%     160276 ± 24%  numa-meminfo.node3.AnonPages.max
    270.83           -19.9%     217.00 ±  2%  vmstat.io.bi
    456385 ±  3%     -18.0%     374348 ±  2%  vmstat.io.bo
    165973           -18.2%     135713 ±  4%  vmstat.system.cs
     13850 ±  5%     +10.8%      15340 ±  4%  meminfo.Active(anon)
    250973           +15.1%     288881        meminfo.AnonHugePages
     55246 ±  2%     -11.5%      48894 ±  3%  meminfo.Shmem
    812.17 ± 24%    +164.0%       2144 ± 30%  meminfo.Writeback
      2011 ±  5%     +22.2%       2458 ±  6%  turbostat.Bzy_MHz
  1.32e+08 ± 13%     +24.5%  1.643e+08 ±  2%  turbostat.IRQ
      0.35 ±  2%      -0.1        0.28 ±  3%  turbostat.POLL%
     96.62 ±  7%     +23.8%     119.66 ±  4%  turbostat.PkgWatt
      7222 ± 57%     -69.0%       2236 ± 79%  numa-vmstat.node0.nr_shmem
     25.17 ± 78%    +521.2%     156.33 ± 22%  numa-vmstat.node0.nr_writeback
   1120967 ± 92%    +554.2%    7333927 ± 15%  numa-vmstat.node0.numa_hit
   1040437 ± 99%    +598.4%    7266899 ± 15%  numa-vmstat.node0.numa_local
     44.17 ± 91%    +243.0%     151.50 ± 18%  numa-vmstat.node1.nr_writeback
      0.20 ±  5%     +13.0%       0.23 ±  5%  sched_debug.cfs_rq:/.h_nr_running.stddev
      0.20 ±  5%     +12.4%       0.23 ±  5%  sched_debug.cfs_rq:/.nr_running.stddev
    156695 ±  7%     +14.2%     178962 ±  6%  sched_debug.cpu.avg_idle.stddev
    191435 ±  7%     +18.7%     227289 ±  4%  sched_debug.cpu.clock.avg
    191444 ±  7%     +18.7%     227298 ±  4%  sched_debug.cpu.clock.max
    191425 ±  7%     +18.7%     227279 ±  4%  sched_debug.cpu.clock.min
    189015 ±  7%     +19.1%     225057 ±  4%  sched_debug.cpu.clock_task.avg
    190352 ±  7%     +18.7%     225903 ±  4%  sched_debug.cpu.clock_task.max
    179649 ±  8%     +20.1%     215848 ±  4%  sched_debug.cpu.clock_task.min
      1498 ±  2%     +10.6%       1657 ±  4%  sched_debug.cpu.curr->pid.stddev
    375793 ±  9%     -21.4%     295204 ± 17%  sched_debug.cpu.nr_switches.stddev
    191425 ±  7%     +18.7%     227279 ±  4%  sched_debug.cpu_clk
    190413 ±  7%     +18.8%     226267 ±  4%  sched_debug.ktime
    192909 ±  7%     +18.0%     227721 ±  4%  sched_debug.sched_clk
      3467 ±  4%     +10.5%       3831 ±  4%  proc-vmstat.nr_active_anon
    102749            -2.0%     100660        proc-vmstat.nr_inactive_anon
     11331            -6.1%      10642 ±  2%  proc-vmstat.nr_mapped
     13948 ±  3%     -11.8%      12308 ±  3%  proc-vmstat.nr_shmem
    127329            +1.9%     129714        proc-vmstat.nr_slab_unreclaimable
    199.67 ± 30%    +163.0%     525.17 ± 30%  proc-vmstat.nr_writeback
      3467 ±  4%     +10.5%       3831 ±  4%  proc-vmstat.nr_zone_active_anon
    102750            -2.0%     100660        proc-vmstat.nr_zone_inactive_anon
  16907009 ± 16%     -88.8%    1888777 ±116%  proc-vmstat.numa_foreign
  15129017 ± 18%     +99.0%   30109594 ±  7%  proc-vmstat.numa_hit
    649.67 ±  8%     +59.0%       1032 ±  6%  proc-vmstat.numa_huge_pte_updates
  14859248 ± 19%    +100.8%   29841270 ±  7%  proc-vmstat.numa_local
  16909166 ± 16%     -88.8%    1888573 ±116%  proc-vmstat.numa_miss
  17173636 ± 16%     -87.5%    2150108 ±102%  proc-vmstat.numa_other
    516559 ±  5%     +10.8%     572525 ±  5%  proc-vmstat.numa_pte_updates
    679842            -5.4%     643129        proc-vmstat.pgactivate
   1688502            +4.1%    1757298        proc-vmstat.pgfault
    109978 ±  3%     +18.0%     129757 ±  2%  proc-vmstat.pgreuse
 1.724e+09           -15.5%  1.457e+09        perf-stat.i.branch-instructions
  23204655 ±  4%     +57.8%   36615017 ±  4%  perf-stat.i.cache-misses
    167616           -18.4%     136728 ±  4%  perf-stat.i.context-switches
      1.99 ±  7%     +25.3%       2.49 ±  6%  perf-stat.i.cpi
    218.69            +2.6%     224.39        perf-stat.i.cpu-migrations
    774.80 ±  3%     -32.8%     520.33 ±  4%  perf-stat.i.cycles-between-cache-misses
 2.125e+09           -16.7%  1.771e+09 ±  2%  perf-stat.i.dTLB-loads
 1.013e+09 ±  2%     -16.0%  8.514e+08 ±  2%  perf-stat.i.dTLB-stores
   4945258 ±  2%      -9.1%    4493474 ±  3%  perf-stat.i.iTLB-load-misses
  7.99e+09           -15.2%  6.778e+09        perf-stat.i.instructions
      1686 ±  2%      -6.1%       1584 ±  2%  perf-stat.i.instructions-per-iTLB-miss
      0.52 ±  6%     -21.1%       0.41 ±  5%  perf-stat.i.ipc
      1.42 ± 24%     -26.9%       1.04 ±  2%  perf-stat.i.major-faults
     25.47           -16.6%      21.24 ±  2%  perf-stat.i.metric.M/sec
      4971 ±  2%     -14.5%       4248        perf-stat.i.minor-faults
     71.55 ±  5%     +17.6       89.13        perf-stat.i.node-load-miss-rate%
   2873999 ± 21%    +224.2%    9317253 ±  9%  perf-stat.i.node-load-misses
     70.57 ±  4%      +8.6       79.15        perf-stat.i.node-store-miss-rate%
   1582580 ± 11%    +116.9%    3432781 ±  6%  perf-stat.i.node-store-misses
    640907 ± 18%     +38.8%     889366 ±  6%  perf-stat.i.node-stores
      4972 ±  2%     -14.5%       4249        perf-stat.i.page-faults
     22.62 ± 21%     +19.2       41.84 ± 22%  perf-stat.overall.cache-miss-rate%
      1.92 ±  6%     +29.0%       2.47 ±  5%  perf-stat.overall.cpi
    659.79 ±  3%     -30.5%     458.63 ±  6%  perf-stat.overall.cycles-between-cache-misses
      1616 ±  2%      -6.6%       1509 ±  3%  perf-stat.overall.instructions-per-iTLB-miss
      0.52 ±  5%     -22.6%       0.41 ±  5%  perf-stat.overall.ipc
     76.92 ±  3%     +12.9       89.80        perf-stat.overall.node-load-miss-rate%
     71.33 ±  3%      +8.1       79.41        perf-stat.overall.node-store-miss-rate%
 1.718e+09           -15.4%  1.453e+09        perf-stat.ps.branch-instructions
  23130416 ±  4%     +57.8%   36508107 ±  4%  perf-stat.ps.cache-misses
    166999           -18.4%     136299 ±  4%  perf-stat.ps.context-switches
    218.01            +2.6%     223.77        perf-stat.ps.cpu-migrations
 2.118e+09           -16.6%  1.765e+09 ±  2%  perf-stat.ps.dTLB-loads
  1.01e+09 ±  2%     -15.9%  8.491e+08 ±  2%  perf-stat.ps.dTLB-stores
   4927968 ±  2%      -9.1%    4480218 ±  3%  perf-stat.ps.iTLB-load-misses
 7.963e+09           -15.1%  6.758e+09        perf-stat.ps.instructions
      1.42 ± 24%     -26.8%       1.04 ±  2%  perf-stat.ps.major-faults
      4952 ±  2%     -14.5%       4234        perf-stat.ps.minor-faults
   2866096 ± 21%    +224.1%    9289790 ±  9%  perf-stat.ps.node-load-misses
   1577533 ± 11%    +117.0%    3422669 ±  6%  perf-stat.ps.node-store-misses
    638821 ± 18%     +38.8%     886869 ±  6%  perf-stat.ps.node-stores
      4953 ±  2%     -14.5%       4235        perf-stat.ps.page-faults
 2.473e+12 ±  2%      +3.8%  2.568e+12        perf-stat.total.instructions
      7.80 ± 10%      -1.5        6.32 ±  9%  perf-profile.calltrace.cycles-pp.poll_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
      2.14 ± 10%      -1.2        0.98 ± 10%  perf-profile.calltrace.cycles-pp.pagevec_lookup_range_tag.f2fs_fsync_node_pages.f2fs_do_sync_file.__x64_sys_fsync.do_syscall_64
      2.13 ±  9%      -1.2        0.97 ± 10%  perf-profile.calltrace.cycles-pp.find_get_pages_range_tag.pagevec_lookup_range_tag.f2fs_fsync_node_pages.f2fs_do_sync_file.__x64_sys_fsync
      1.54 ±  8%      -0.6        0.96 ± 37%  perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_stat_runtime
      2.05 ± 10%      -0.5        1.51 ± 12%  perf-profile.calltrace.cycles-pp.generic_perform_write.f2fs_buffered_write_iter.f2fs_file_write_iter.new_sync_write.vfs_write
      2.06 ± 10%      -0.5        1.54 ± 12%  perf-profile.calltrace.cycles-pp.f2fs_buffered_write_iter.f2fs_file_write_iter.new_sync_write.vfs_write.ksys_write
      2.12 ±  8%      -0.5        1.61 ± 13%  perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.io_schedule
      2.15 ±  8%      -0.5        1.65 ± 13%  perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.io_schedule.folio_wait_bit_common
      1.73 ±  8%      -0.5        1.25 ± 25%  perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity
      1.72 ±  8%      -0.5        1.24 ± 24%  perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr
      1.94 ±  7%      -0.5        1.47 ± 14%  perf-profile.calltrace.cycles-pp.schedule.io_schedule.folio_wait_bit_common.folio_wait_writeback.f2fs_wait_on_page_writeback
      1.94 ±  7%      -0.5        1.48 ± 14%  perf-profile.calltrace.cycles-pp.io_schedule.folio_wait_bit_common.folio_wait_writeback.f2fs_wait_on_page_writeback.f2fs_wait_on_node_pages_writeback
      2.03 ±  6%      -0.5        1.57 ± 12%  perf-profile.calltrace.cycles-pp.folio_wait_bit_common.folio_wait_writeback.f2fs_wait_on_page_writeback.f2fs_wait_on_node_pages_writeback.f2fs_do_sync_file
      2.05 ±  6%      -0.5        1.59 ± 12%  perf-profile.calltrace.cycles-pp.folio_wait_writeback.f2fs_wait_on_page_writeback.f2fs_wait_on_node_pages_writeback.f2fs_do_sync_file.__x64_sys_fsync
      1.92 ±  8%      -0.5        1.46 ± 14%  perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__schedule.schedule
      1.80 ±  8%      -0.4        1.39 ± 14%  perf-profile.calltrace.cycles-pp.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair.__schedule
      1.77 ±  8%      -0.4        1.36 ± 14%  perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair
      0.67 ± 19%      -0.4        0.30 ±100%  perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork
      0.86 ± 14%      -0.3        0.55 ± 47%  perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.locked_inode_to_wb_and_lock_list.__mark_inode_dirty.f2fs_update_parent_metadata
      0.89 ± 14%      -0.2        0.67 ± 16%  perf-profile.calltrace.cycles-pp._raw_spin_lock.locked_inode_to_wb_and_lock_list.__mark_inode_dirty.f2fs_update_parent_metadata.f2fs_add_regular_entry
      0.89 ± 13%      -0.2        0.69 ± 14%  perf-profile.calltrace.cycles-pp.locked_inode_to_wb_and_lock_list.__mark_inode_dirty.f2fs_update_parent_metadata.f2fs_add_regular_entry.f2fs_add_dentry
      0.76 ±  7%      +0.2        0.95 ± 11%  perf-profile.calltrace.cycles-pp.__percpu_counter_sum.f2fs_space_for_roll_forward.f2fs_do_sync_file.__x64_sys_fsync.do_syscall_64
      0.80 ±  6%      +0.2        1.02 ± 11%  perf-profile.calltrace.cycles-pp.f2fs_space_for_roll_forward.f2fs_do_sync_file.__x64_sys_fsync.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.08 ± 16%      +0.4        1.43 ± 11%  perf-profile.calltrace.cycles-pp.__softirqentry_text_start.__irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
      1.60 ± 10%      +0.5        2.06 ±  7%  perf-profile.calltrace.cycles-pp.f2fs_new_inode_page.f2fs_init_inode_metadata.f2fs_add_regular_entry.f2fs_add_dentry.f2fs_do_add_link
      1.59 ± 10%      +0.5        2.06 ±  7%  perf-profile.calltrace.cycles-pp.f2fs_new_node_page.f2fs_new_inode_page.f2fs_init_inode_metadata.f2fs_add_regular_entry.f2fs_add_dentry
      0.29 ±100%      +0.5        0.76 ± 11%  perf-profile.calltrace.cycles-pp.rebalance_domains.__softirqentry_text_start.__irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      3.68 ±  6%      +0.5        4.16 ±  8%  perf-profile.calltrace.cycles-pp.f2fs_do_add_link.f2fs_create.lookup_open.open_last_lookups.path_openat
      0.79 ± 11%      +0.5        1.27 ± 11%  perf-profile.calltrace.cycles-pp.f2fs_preallocate_blocks.f2fs_file_write_iter.new_sync_write.vfs_write.ksys_write
      3.56 ±  7%      +0.5        4.07 ±  8%  perf-profile.calltrace.cycles-pp.f2fs_add_regular_entry.f2fs_add_dentry.f2fs_do_add_link.f2fs_create.lookup_open
      3.56 ±  7%      +0.5        4.07 ±  8%  perf-profile.calltrace.cycles-pp.f2fs_add_dentry.f2fs_do_add_link.f2fs_create.lookup_open.open_last_lookups
      0.86 ± 24%      +0.5        1.39 ± 29%  perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
      1.81 ± 11%      +0.6        2.36 ±  8%  perf-profile.calltrace.cycles-pp.f2fs_init_inode_metadata.f2fs_add_regular_entry.f2fs_add_dentry.f2fs_do_add_link.f2fs_create
      0.18 ±141%      +0.6        0.75 ± 10%  perf-profile.calltrace.cycles-pp.f2fs_convert_inline_inode.f2fs_preallocate_blocks.f2fs_file_write_iter.new_sync_write.vfs_write
      0.74 ± 25%      +0.6        1.36 ±  8%  perf-profile.calltrace.cycles-pp.f2fs_do_write_node_page.__write_node_page.f2fs_fsync_node_pages.f2fs_do_sync_file.__x64_sys_fsync
      0.74 ± 25%      +0.6        1.36 ±  8%  perf-profile.calltrace.cycles-pp.do_write_page.f2fs_do_write_node_page.__write_node_page.f2fs_fsync_node_pages.f2fs_do_sync_file
      0.00            +0.6        0.63 ±  6%  perf-profile.calltrace.cycles-pp.set_node_addr.f2fs_new_node_page.f2fs_new_inode_page.f2fs_init_inode_metadata.f2fs_add_regular_entry
      0.10 ±223%      +0.7        0.77 ±  8%  perf-profile.calltrace.cycles-pp.f2fs_allocate_data_block.do_write_page.f2fs_do_write_node_page.__write_node_page.f2fs_fsync_node_pages
      0.00            +0.7        0.68 ±  8%  perf-profile.calltrace.cycles-pp.f2fs_submit_page_write.do_write_page.f2fs_outplace_write_data.f2fs_do_write_data_page.f2fs_write_single_data_page
      0.53 ± 52%      +0.7        1.26 ± 11%  perf-profile.calltrace.cycles-pp.f2fs_allocate_data_block.do_write_page.f2fs_outplace_write_data.f2fs_do_write_data_page.f2fs_write_single_data_page
      1.48 ± 20%      +0.8        2.32 ± 25%  perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
      1.42 ± 21%      +0.8        2.26 ± 26%  perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt
      5.87 ±  7%      +1.0        6.85 ±  8%  perf-profile.calltrace.cycles-pp.f2fs_create.lookup_open.open_last_lookups.path_openat.do_filp_open
      1.10 ± 20%      +1.0        2.10 ±  9%  perf-profile.calltrace.cycles-pp.do_write_page.f2fs_outplace_write_data.f2fs_do_write_data_page.f2fs_write_single_data_page.f2fs_write_cache_pages
      0.66 ± 58%      +1.0        1.67 ± 35%  perf-profile.calltrace.cycles-pp.timekeeping_max_deferment.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.cpuidle_idle_call
      1.42 ± 18%      +1.1        2.52 ±  9%  perf-profile.calltrace.cycles-pp.f2fs_outplace_write_data.f2fs_do_write_data_page.f2fs_write_single_data_page.f2fs_write_cache_pages.__f2fs_write_data_pages
      2.74 ± 22%      +1.2        3.96 ± 18%  perf-profile.calltrace.cycles-pp.menu_select.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
      1.46 ± 23%      +1.2        2.69 ±  7%  perf-profile.calltrace.cycles-pp.__write_node_page.f2fs_fsync_node_pages.f2fs_do_sync_file.__x64_sys_fsync.do_syscall_64
      1.48 ± 30%      +1.3        2.73 ± 30%  perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.cpuidle_idle_call.do_idle.cpu_startup_entry
      1.23 ± 34%      +1.3        2.50 ± 33%  perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.cpuidle_idle_call.do_idle
      0.19 ±223%      +1.3        1.53 ± 34%  perf-profile.calltrace.cycles-pp.handle_edge_irq.__common_interrupt.common_interrupt.asm_common_interrupt.poll_idle
      0.19 ±223%      +1.3        1.53 ± 34%  perf-profile.calltrace.cycles-pp.__common_interrupt.common_interrupt.asm_common_interrupt.poll_idle.cpuidle_enter_state
      0.20 ±223%      +1.4        1.57 ± 34%  perf-profile.calltrace.cycles-pp.common_interrupt.asm_common_interrupt.poll_idle.cpuidle_enter_state.cpuidle_enter
      0.20 ±223%      +1.4        1.58 ± 34%  perf-profile.calltrace.cycles-pp.asm_common_interrupt.poll_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call
      4.62 ± 20%      +1.4        6.04 ± 10%  perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
      4.52 ± 21%      +1.4        5.96 ± 11%  perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
      6.28 ±  8%      +1.6        7.90 ± 10%  perf-profile.calltrace.cycles-pp.file_write_and_wait_range.f2fs_do_sync_file.__x64_sys_fsync.do_syscall_64.entry_SYSCALL_64_after_hwframe
      2.15 ± 19%      +1.7        3.87 ±  9%  perf-profile.calltrace.cycles-pp.f2fs_do_write_data_page.f2fs_write_single_data_page.f2fs_write_cache_pages.__f2fs_write_data_pages.do_writepages
      2.38 ± 18%      +1.8        4.20 ±  9%  perf-profile.calltrace.cycles-pp.f2fs_write_single_data_page.f2fs_write_cache_pages.__f2fs_write_data_pages.do_writepages.filemap_fdatawrite_wbc
      3.78 ± 14%      +2.0        5.79 ± 10%  perf-profile.calltrace.cycles-pp.do_writepages.filemap_fdatawrite_wbc.file_write_and_wait_range.f2fs_do_sync_file.__x64_sys_fsync
      3.78 ± 14%      +2.0        5.80 ± 10%  perf-profile.calltrace.cycles-pp.filemap_fdatawrite_wbc.file_write_and_wait_range.f2fs_do_sync_file.__x64_sys_fsync.do_syscall_64
      3.40 ± 16%      +2.0        5.42 ±  9%  perf-profile.calltrace.cycles-pp.f2fs_write_cache_pages.__f2fs_write_data_pages.do_writepages.filemap_fdatawrite_wbc.file_write_and_wait_range
      3.76 ± 15%      +2.0        5.78 ± 10%  perf-profile.calltrace.cycles-pp.__f2fs_write_data_pages.do_writepages.filemap_fdatawrite_wbc.file_write_and_wait_range.f2fs_do_sync_file
      7.22 ± 20%      +2.3        9.48 ±  9%  perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call
      7.82 ± 10%      -1.5        6.36 ±  9%  perf-profile.children.cycles-pp.poll_idle
      2.23 ±  9%      -1.2        1.05 ±  9%  perf-profile.children.cycles-pp.pagevec_lookup_range_tag
      2.22 ±  9%      -1.2        1.05 ±  9%  perf-profile.children.cycles-pp.find_get_pages_range_tag
      2.06 ± 10%      -0.5        1.54 ± 12%  perf-profile.children.cycles-pp.f2fs_buffered_write_iter
      2.43 ± 10%      -0.5        1.94 ± 12%  perf-profile.children.cycles-pp.generic_perform_write
      2.40 ±  5%      -0.4        1.98 ± 12%  perf-profile.children.cycles-pp.dequeue_entity
      0.95 ± 16%      -0.4        0.60 ± 16%  perf-profile.children.cycles-pp.copy_page_from_iter_atomic
      0.93 ± 16%      -0.3        0.58 ± 17%  perf-profile.children.cycles-pp.copyin
      0.93 ± 16%      -0.3        0.59 ± 16%  perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
      0.70 ±  6%      -0.3        0.44 ±  3%  perf-profile.children.cycles-pp.xas_find_marked
      0.52 ±  9%      -0.2        0.30 ± 22%  perf-profile.children.cycles-pp.__pagevec_release
      0.92 ± 13%      -0.2        0.70 ± 14%  perf-profile.children.cycles-pp.locked_inode_to_wb_and_lock_list
      0.67 ± 19%      -0.2        0.51 ± 20%  perf-profile.children.cycles-pp.worker_thread
      0.49 ± 10%      -0.1        0.35 ± 10%  perf-profile.children.cycles-pp.alloc_inode
      0.55 ± 12%      -0.1        0.41 ± 10%  perf-profile.children.cycles-pp.new_inode
      0.36 ± 10%      -0.1        0.23 ± 14%  perf-profile.children.cycles-pp.__alloc_pages
      0.59 ±  6%      -0.1        0.46 ±  8%  perf-profile.children.cycles-pp.kmem_cache_alloc_lru
      0.49 ± 10%      -0.1        0.36 ± 12%  perf-profile.children.cycles-pp.kmem_cache_alloc
      0.30 ± 15%      -0.1        0.17 ± 14%  perf-profile.children.cycles-pp.___slab_alloc
      0.28 ± 27%      -0.1        0.16 ± 21%  perf-profile.children.cycles-pp.release_pages
      0.30 ± 10%      -0.1        0.18 ± 19%  perf-profile.children.cycles-pp.folio_alloc
      0.30 ± 10%      -0.1        0.19 ± 16%  perf-profile.children.cycles-pp.get_page_from_freelist
      0.23 ± 20%      -0.1        0.12 ± 13%  perf-profile.children.cycles-pp.allocate_slab
      0.43 ±  6%      -0.1        0.32 ±  9%  perf-profile.children.cycles-pp.f2fs_alloc_inode
      0.37 ± 12%      -0.1        0.27 ± 18%  perf-profile.children.cycles-pp.ttwu_do_activate
      0.25 ±  6%      -0.1        0.16 ± 10%  perf-profile.children.cycles-pp.memcg_slab_post_alloc_hook
      0.25 ± 14%      -0.1        0.16 ± 23%  perf-profile.children.cycles-pp.lru_add_drain_cpu
      0.25 ± 14%      -0.1        0.17 ± 21%  perf-profile.children.cycles-pp.__pagevec_lru_add
      0.27 ± 12%      -0.1        0.19 ± 17%  perf-profile.children.cycles-pp.xas_create
      0.19 ± 23%      -0.1        0.11 ± 29%  perf-profile.children.cycles-pp.nvme_poll_cq
      0.14 ± 27%      -0.1        0.07 ± 13%  perf-profile.children.cycles-pp.setup_object
      0.24 ± 11%      -0.1        0.17 ± 16%  perf-profile.children.cycles-pp.xas_expand
      0.21 ± 14%      -0.1        0.14 ± 22%  perf-profile.children.cycles-pp.mempool_alloc
      0.20 ± 15%      -0.1        0.13 ± 25%  perf-profile.children.cycles-pp.__pagevec_lru_add_fn
      0.21 ± 10%      -0.1        0.14 ± 19%  perf-profile.children.cycles-pp.rmqueue
      0.36 ± 12%      -0.1        0.29 ±  9%  perf-profile.children.cycles-pp.update_rq_clock
      0.12 ± 16%      -0.1        0.04 ± 45%  perf-profile.children.cycles-pp.inode_init_once
      0.16 ± 10%      -0.1        0.10 ± 32%  perf-profile.children.cycles-pp.set_next_entity
      0.24 ± 14%      -0.1        0.18 ± 18%  perf-profile.children.cycles-pp.enqueue_entity
      0.29 ±  8%      -0.1        0.23 ±  4%  perf-profile.children.cycles-pp.d_alloc_parallel
      0.27 ±  8%      -0.1        0.22 ±  5%  perf-profile.children.cycles-pp.d_alloc
      0.15 ± 18%      -0.0        0.11 ± 16%  perf-profile.children.cycles-pp.select_task_rq
      0.07 ± 14%      -0.0        0.03 ±100%  perf-profile.children.cycles-pp.syscall_return_via_sysret
      0.30 ±  3%      -0.0        0.27 ±  8%  perf-profile.children.cycles-pp.exc_page_fault
      0.08 ±  8%      -0.0        0.04 ± 72%  perf-profile.children.cycles-pp.filp_close
      0.12 ±  7%      -0.0        0.09 ± 10%  perf-profile.children.cycles-pp.handle_mm_fault
      0.12 ± 10%      -0.0        0.08 ±  8%  perf-profile.children.cycles-pp.__handle_mm_fault
      0.15 ±  7%      -0.0        0.12 ±  5%  perf-profile.children.cycles-pp.do_user_addr_fault
      0.09 ± 15%      -0.0        0.06 ± 19%  perf-profile.children.cycles-pp.rmqueue_bulk
      0.10 ±  6%      -0.0        0.07 ± 17%  perf-profile.children.cycles-pp.perf_output_begin_forward
      0.09 ± 27%      -0.0        0.06 ± 23%  perf-profile.children.cycles-pp.iostat_update_and_unbind_ctx
      0.12 ± 18%      -0.0        0.10 ± 11%  perf-profile.children.cycles-pp.select_task_rq_fair
      0.16 ± 10%      -0.0        0.13 ±  9%  perf-profile.children.cycles-pp.walk_component
      0.13 ± 13%      +0.0        0.16 ±  6%  perf-profile.children.cycles-pp.wbt_wait
      0.12 ± 12%      +0.0        0.15 ±  6%  perf-profile.children.cycles-pp.rq_qos_wait
      0.14 ± 14%      +0.0        0.17 ±  5%  perf-profile.children.cycles-pp.__rq_qos_throttle
      0.07 ± 20%      +0.0        0.11 ± 22%  perf-profile.children.cycles-pp.xas_set_mark
      0.14 ±  9%      +0.0        0.18 ±  9%  perf-profile.children.cycles-pp.do_dentry_open
      0.07 ± 48%      +0.0        0.11 ±  9%  perf-profile.children.cycles-pp.wbt_rqw_done
      0.05 ± 46%      +0.0        0.09 ± 19%  perf-profile.children.cycles-pp.security_file_open
      0.07 ± 46%      +0.0        0.12 ±  9%  perf-profile.children.cycles-pp.wbt_done
      0.07 ± 47%      +0.0        0.12 ± 11%  perf-profile.children.cycles-pp.__rq_qos_done
      0.04 ± 73%      +0.0        0.08 ± 11%  perf-profile.children.cycles-pp.queue_delayed_work_on
      0.10 ± 12%      +0.0        0.15 ± 13%  perf-profile.children.cycles-pp.f2fs_del_fsync_node_entry
      0.05 ± 46%      +0.0        0.10 ± 17%  perf-profile.children.cycles-pp.folio_add_lru
      0.08 ± 20%      +0.1        0.14 ± 15%  perf-profile.children.cycles-pp.__xa_clear_mark
      0.02 ±142%      +0.1        0.07 ± 14%  perf-profile.children.cycles-pp.apparmor_file_alloc_security
      0.01 ±223%      +0.1        0.06 ± 14%  perf-profile.children.cycles-pp.rwsem_mark_wake
      0.11 ± 10%      +0.1        0.16 ± 13%  perf-profile.children.cycles-pp.__set_nat_cache_dirty
      0.03 ±100%      +0.1        0.09 ± 20%  perf-profile.children.cycles-pp.apparmor_file_open
      0.05 ± 48%      +0.1        0.12 ±  9%  perf-profile.children.cycles-pp.f2fs_convert_inline_page
      0.00            +0.1        0.06 ± 14%  perf-profile.children.cycles-pp.f2fs_submit_merged_ipu_write
      0.18 ± 19%      +0.1        0.25 ± 19%  perf-profile.children.cycles-pp.__intel_pmu_enable_all
      0.05 ± 46%      +0.1        0.12 ± 13%  perf-profile.children.cycles-pp.ttwu_queue_wakelist
      0.03 ±100%      +0.1        0.10 ± 16%  perf-profile.children.cycles-pp.__remove_ino_entry
      0.10 ± 31%      +0.1        0.17 ± 27%  perf-profile.children.cycles-pp.idle_cpu
      0.08 ± 34%      +0.1        0.15 ± 10%  perf-profile.children.cycles-pp.sb_mark_inode_writeback
      0.09 ± 26%      +0.1        0.16 ± 11%  perf-profile.children.cycles-pp.inc_valid_block_count
      0.04 ± 45%      +0.1        0.12 ± 11%  perf-profile.children.cycles-pp.f2fs_need_inode_block_update
      0.11 ± 18%      +0.1        0.18 ± 13%  perf-profile.children.cycles-pp.f2fs_try_to_free_nats
      0.00            +0.1        0.08 ± 12%  perf-profile.children.cycles-pp.__flush_smp_call_function_queue
      0.00            +0.1        0.08 ± 17%  perf-profile.children.cycles-pp.__get_segment_type
      0.20 ± 21%      +0.1        0.27 ± 17%  perf-profile.children.cycles-pp.f2fs_init_acl
      0.07 ± 24%      +0.1        0.15 ± 17%  perf-profile.children.cycles-pp.__xa_set_mark
      0.01 ±223%      +0.1        0.09 ±  9%  perf-profile.children.cycles-pp.inc_valid_node_count
      0.17 ± 15%      +0.1        0.25 ± 12%  perf-profile.children.cycles-pp.f2fs_is_checkpointed_node
      0.18 ± 24%      +0.1        0.26 ± 16%  perf-profile.children.cycles-pp.__f2fs_get_acl
      0.05 ± 55%      +0.1        0.14 ± 11%  perf-profile.children.cycles-pp.sb_clear_inode_writeback
      0.00            +0.1        0.08 ± 13%  perf-profile.children.cycles-pp.f2fs_lookup_extent_cache
      0.20 ± 12%      +0.1        0.28 ±  9%  perf-profile.children.cycles-pp.f2fs_update_inode
      0.14 ± 19%      +0.1        0.22 ± 15%  perf-profile.children.cycles-pp.f2fs_reserve_new_blocks
      0.17 ± 26%      +0.1        0.26 ± 16%  perf-profile.children.cycles-pp.f2fs_getxattr
      0.16 ± 26%      +0.1        0.24 ± 16%  perf-profile.children.cycles-pp.lookup_all_xattrs
      0.12 ± 17%      +0.1        0.21 ±  5%  perf-profile.children.cycles-pp.f2fs_inode_synced
      0.08 ± 26%      +0.1        0.16 ± 13%  perf-profile.children.cycles-pp.sched_ttwu_pending
      0.02 ±143%      +0.1        0.11 ±  9%  perf-profile.children.cycles-pp.__init_nat_entry
      0.16 ± 10%      +0.1        0.25 ± 22%  perf-profile.children.cycles-pp.__is_cp_guaranteed
      0.09 ± 15%      +0.1        0.18 ± 14%  perf-profile.children.cycles-pp.f2fs_alloc_nid_done
      0.21 ± 18%      +0.1        0.30 ±  6%  perf-profile.children.cycles-pp.__grab_extent_tree
      0.11 ± 19%      +0.1        0.20 ± 16%  perf-profile.children.cycles-pp.folio_account_dirtied
      0.07 ± 23%      +0.1        0.18 ± 10%  perf-profile.children.cycles-pp.f2fs_alloc_nid
      0.02 ±149%      +0.1        0.13 ±  9%  perf-profile.children.cycles-pp.f2fs_is_valid_blkaddr
      0.09 ± 31%      +0.1        0.20 ± 19%  perf-profile.children.cycles-pp.update_segment_mtime
      0.22 ± 20%      +0.1        0.33 ±  8%  perf-profile.children.cycles-pp.f2fs_init_extent_tree
      0.13 ± 23%      +0.1        0.25 ± 21%  perf-profile.children.cycles-pp.percpu_counter_add_batch
      0.10 ± 45%      +0.1        0.23 ± 14%  perf-profile.children.cycles-pp.osq_lock
      0.24 ± 18%      +0.1        0.37 ±  8%  perf-profile.children.cycles-pp.f2fs_mark_inode_dirty_sync
      0.28 ± 21%      +0.1        0.41 ±  4%  perf-profile.children.cycles-pp.wake_up_q
      0.15 ± 14%      +0.1        0.30 ±  6%  perf-profile.children.cycles-pp.f2fs_update_dirty_folio
      0.27 ± 15%      +0.2        0.42 ±  4%  perf-profile.children.cycles-pp.mutex_lock
      0.30 ±  8%      +0.2        0.45 ±  6%  perf-profile.children.cycles-pp.up_read
      0.26 ± 20%      +0.2        0.41 ±  7%  perf-profile.children.cycles-pp.f2fs_inode_dirtied
      0.21 ± 40%      +0.2        0.38 ± 11%  perf-profile.children.cycles-pp.f2fs_get_read_data_page
      0.30 ± 23%      +0.2        0.47 ±  3%  perf-profile.children.cycles-pp.rwsem_wake
      0.01 ±223%      +0.2        0.18 ± 18%  perf-profile.children.cycles-pp.flush_smp_call_function_queue
      0.78 ±  6%      +0.2        0.96 ± 11%  perf-profile.children.cycles-pp.__percpu_counter_sum
      0.05 ± 94%      +0.2        0.24 ± 16%  perf-profile.children.cycles-pp.f2fs_inode_chksum_verify
      0.19 ± 37%      +0.2        0.39 ±  9%  perf-profile.children.cycles-pp.rwsem_spin_on_owner
      0.23 ± 22%      +0.2        0.44 ± 14%  perf-profile.children.cycles-pp.f2fs_dirty_node_folio
      0.24 ± 19%      +0.2        0.45 ± 15%  perf-profile.children.cycles-pp.__folio_mark_dirty
      0.20 ± 26%      +0.2        0.42 ±  9%  perf-profile.children.cycles-pp.__lookup_nat_cache
      0.18 ± 37%      +0.2        0.39 ± 15%  perf-profile.children.cycles-pp.update_sit_entry
      0.09 ± 57%      +0.2        0.30 ± 13%  perf-profile.children.cycles-pp.has_not_enough_free_secs
      0.29 ± 18%      +0.2        0.51 ± 11%  perf-profile.children.cycles-pp.f2fs_update_inode_page
      0.29 ± 23%      +0.2        0.50 ± 12%  perf-profile.children.cycles-pp._raw_spin_trylock
      0.27 ± 21%      +0.2        0.49 ± 13%  perf-profile.children.cycles-pp.f2fs_map_blocks
      0.16 ± 33%      +0.2        0.38 ± 14%  perf-profile.children.cycles-pp.read_node_page
      0.80 ±  6%      +0.2        1.03 ± 11%  perf-profile.children.cycles-pp.f2fs_space_for_roll_forward
      0.68 ±  9%      +0.2        0.91 ± 12%  perf-profile.children.cycles-pp.xas_load
      0.32 ± 19%      +0.2        0.56 ± 11%  perf-profile.children.cycles-pp.f2fs_write_inode
      0.32 ± 24%      +0.2        0.56 ±  8%  perf-profile.children.cycles-pp.down_write
      0.28 ± 21%      +0.3        0.53 ±  6%  perf-profile.children.cycles-pp.__radix_tree_lookup
      0.37 ± 10%      +0.3        0.63 ±  8%  perf-profile.children.cycles-pp.f2fs_dirty_data_folio
      0.48 ±  9%      +0.3        0.76 ± 11%  perf-profile.children.cycles-pp.f2fs_convert_inline_inode
      0.38 ± 17%      +0.3        0.68 ± 13%  perf-profile.children.cycles-pp.filemap_dirty_folio
      0.46 ± 16%      +0.3        0.77 ±  9%  perf-profile.children.cycles-pp.__folio_end_writeback
      0.34 ± 22%      +0.4        0.70 ±  8%  perf-profile.children.cycles-pp.f2fs_get_node_info
      0.69 ± 18%      +0.4        1.07 ± 11%  perf-profile.children.cycles-pp.f2fs_get_dnode_of_data
      0.29 ± 36%      +0.4        0.67 ± 12%  perf-profile.children.cycles-pp.f2fs_balance_fs
      0.63 ± 10%      +0.4        1.04 ±  7%  perf-profile.children.cycles-pp.down_read
      0.40 ± 30%      +0.4        0.82 ± 11%  perf-profile.children.cycles-pp.rwsem_optimistic_spin
      0.54 ± 17%      +0.4        0.97 ±  7%  perf-profile.children.cycles-pp.set_node_addr
      0.42 ± 31%      +0.4        0.87 ± 11%  perf-profile.children.cycles-pp.rwsem_down_write_slowpath
      0.86 ± 12%      +0.5        1.32 ±  5%  perf-profile.children.cycles-pp._raw_spin_lock_irqsave
      1.60 ± 10%      +0.5        2.06 ±  7%  perf-profile.children.cycles-pp.f2fs_new_inode_page
      1.59 ± 10%      +0.5        2.06 ±  7%  perf-profile.children.cycles-pp.f2fs_new_node_page
      0.72 ± 17%      +0.5        1.19 ±  8%  perf-profile.children.cycles-pp.f2fs_submit_page_write
      3.68 ±  6%      +0.5        4.16 ±  8%  perf-profile.children.cycles-pp.f2fs_do_add_link
      0.79 ± 11%      +0.5        1.27 ± 11%  perf-profile.children.cycles-pp.f2fs_preallocate_blocks
      3.56 ±  7%      +0.5        4.07 ±  8%  perf-profile.children.cycles-pp.f2fs_add_regular_entry
      3.56 ±  7%      +0.5        4.07 ±  8%  perf-profile.children.cycles-pp.f2fs_add_dentry
      1.81 ± 11%      +0.6        2.36 ±  8%  perf-profile.children.cycles-pp.f2fs_init_inode_metadata
      0.74 ± 25%      +0.6        1.36 ±  8%  perf-profile.children.cycles-pp.f2fs_do_write_node_page
      0.90 ± 19%      +0.7        1.56 ± 11%  perf-profile.children.cycles-pp.__get_node_page
      0.71 ± 42%      +1.0        1.68 ± 35%  perf-profile.children.cycles-pp.timekeeping_max_deferment
      5.88 ±  7%      +1.0        6.85 ±  8%  perf-profile.children.cycles-pp.f2fs_create
      1.13 ± 23%      +1.0        2.13 ±  9%  perf-profile.children.cycles-pp.f2fs_allocate_data_block
      1.60 ± 16%      +1.1        2.67 ±  8%  perf-profile.children.cycles-pp.f2fs_outplace_write_data
      1.48 ± 23%      +1.2        2.70 ±  7%  perf-profile.children.cycles-pp.__write_node_page
      2.77 ± 22%      +1.2        4.00 ± 18%  perf-profile.children.cycles-pp.menu_select
      5.21 ± 16%      +1.3        6.46 ±  9%  perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
      1.49 ± 30%      +1.3        2.76 ± 30%  perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
      1.24 ± 33%      +1.3        2.52 ± 33%  perf-profile.children.cycles-pp.tick_nohz_next_event
      5.12 ± 16%      +1.3        6.40 ±  9%  perf-profile.children.cycles-pp.hrtimer_interrupt
      2.02 ± 19%      +1.6        3.60 ±  8%  perf-profile.children.cycles-pp.do_write_page
      6.28 ±  8%      +1.6        7.91 ± 10%  perf-profile.children.cycles-pp.file_write_and_wait_range
      2.51 ± 16%      +1.7        4.16 ±  8%  perf-profile.children.cycles-pp.f2fs_do_write_data_page
      2.67 ± 17%      +1.8        4.50 ±  8%  perf-profile.children.cycles-pp.f2fs_write_single_data_page
      4.35 ± 13%      +1.9        6.25 ±  9%  perf-profile.children.cycles-pp.do_writepages
      4.26 ± 13%      +1.9        6.19 ±  9%  perf-profile.children.cycles-pp.filemap_fdatawrite_wbc
      3.87 ± 14%      +1.9        5.81 ±  9%  perf-profile.children.cycles-pp.f2fs_write_cache_pages
      4.24 ± 13%      +1.9        6.18 ±  9%  perf-profile.children.cycles-pp.__f2fs_write_data_pages
      8.12 ± 16%      +2.0       10.09 ±  8%  perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
      7.37 ± 12%      -2.7        4.65 ± 20%  perf-profile.self.cycles-pp.poll_idle
      1.52 ± 12%      -0.9        0.62 ± 16%  perf-profile.self.cycles-pp.find_get_pages_range_tag
      0.90 ± 15%      -0.3        0.58 ± 16%  perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
      0.56 ± 21%      -0.3        0.25 ± 13%  perf-profile.self.cycles-pp.f2fs_new_node_page
      0.69 ±  6%      -0.3        0.43 ±  2%  perf-profile.self.cycles-pp.xas_find_marked
      0.40 ± 15%      -0.2        0.19 ± 19%  perf-profile.self.cycles-pp.f2fs_write_end
      0.25 ± 29%      -0.1        0.14 ± 21%  perf-profile.self.cycles-pp.release_pages
      0.11 ± 16%      -0.1        0.04 ± 71%  perf-profile.self.cycles-pp.inode_init_once
      0.18 ± 23%      -0.1        0.11 ± 26%  perf-profile.self.cycles-pp.nvme_poll_cq
      0.24 ±  7%      -0.1        0.18 ± 20%  perf-profile.self.cycles-pp.__schedule
      0.15 ± 11%      -0.1        0.09 ±  6%  perf-profile.self.cycles-pp.memcg_slab_post_alloc_hook
      0.20 ± 12%      -0.1        0.14 ± 19%  perf-profile.self.cycles-pp.kmem_cache_alloc
      0.11 ± 16%      -0.1        0.06 ± 56%  perf-profile.self.cycles-pp.__pagevec_lru_add_fn
      0.14 ± 11%      -0.0        0.10 ± 16%  perf-profile.self.cycles-pp.do_idle
      0.15 ±  8%      -0.0        0.11 ± 16%  perf-profile.self.cycles-pp.update_rq_clock
      0.07 ± 14%      -0.0        0.03 ±100%  perf-profile.self.cycles-pp.syscall_return_via_sysret
      0.09 ± 27%      -0.0        0.06 ± 23%  perf-profile.self.cycles-pp.iostat_update_and_unbind_ctx
      0.10 ±  7%      -0.0        0.07 ± 17%  perf-profile.self.cycles-pp.perf_output_begin_forward
      0.07 ± 15%      +0.0        0.10 ± 15%  perf-profile.self.cycles-pp.page_counter_charge
      0.08 ± 20%      +0.0        0.10 ± 13%  perf-profile.self.cycles-pp.f2fs_do_write_data_page
      0.07 ± 18%      +0.0        0.10 ± 23%  perf-profile.self.cycles-pp.xas_set_mark
      0.06 ± 49%      +0.0        0.10 ±  5%  perf-profile.self.cycles-pp.rq_qos_wait
      0.07 ± 48%      +0.0        0.11 ±  9%  perf-profile.self.cycles-pp.wbt_rqw_done
      0.07 ± 26%      +0.0        0.12 ± 16%  perf-profile.self.cycles-pp.f2fs_write_single_data_page
      0.08 ± 27%      +0.0        0.12 ±  9%  perf-profile.self.cycles-pp.f2fs_wait_on_node_pages_writeback
      0.04 ± 73%      +0.0        0.08 ± 11%  perf-profile.self.cycles-pp.queue_delayed_work_on
      0.11 ± 13%      +0.0        0.16 ± 12%  perf-profile.self.cycles-pp.f2fs_fsync_node_pages
      0.09 ± 23%      +0.1        0.15 ± 12%  perf-profile.self.cycles-pp.__folio_start_writeback
      0.01 ±223%      +0.1        0.06 ± 11%  perf-profile.self.cycles-pp.rwsem_mark_wake
      0.08 ± 22%      +0.1        0.14 ± 14%  perf-profile.self.cycles-pp.read_node_page
      0.08 ± 23%      +0.1        0.14 ± 10%  perf-profile.self.cycles-pp.f2fs_inode_dirtied
      0.06 ± 17%      +0.1        0.12 ± 12%  perf-profile.self.cycles-pp.f2fs_create
      0.01 ±223%      +0.1        0.07 ± 18%  perf-profile.self.cycles-pp.apparmor_file_alloc_security
      0.01 ±223%      +0.1        0.06 ± 14%  perf-profile.self.cycles-pp.__set_nat_cache_dirty
      0.00            +0.1        0.06 ± 19%  perf-profile.self.cycles-pp.f2fs_write_begin
      0.03 ±100%      +0.1        0.09 ± 20%  perf-profile.self.cycles-pp.apparmor_file_open
      0.08 ± 26%      +0.1        0.14 ± 13%  perf-profile.self.cycles-pp.__mark_inode_dirty
      0.00            +0.1        0.06 ± 17%  perf-profile.self.cycles-pp.f2fs_space_for_roll_forward
      0.00            +0.1        0.06 ± 17%  perf-profile.self.cycles-pp.f2fs_convert_inline_inode
      0.00            +0.1        0.06 ± 14%  perf-profile.self.cycles-pp.list_lru_add
      0.00            +0.1        0.06 ± 14%  perf-profile.self.cycles-pp.f2fs_submit_merged_ipu_write
      0.18 ± 19%      +0.1        0.25 ± 19%  perf-profile.self.cycles-pp.__intel_pmu_enable_all
      0.01 ±223%      +0.1        0.08 ± 12%  perf-profile.self.cycles-pp.folio_add_lru
      0.09 ± 13%      +0.1        0.16 ±  3%  perf-profile.self.cycles-pp.f2fs_update_dirty_folio
      0.00            +0.1        0.08 ± 11%  perf-profile.self.cycles-pp.f2fs_lookup_extent_cache
      0.00            +0.1        0.08 ± 10%  perf-profile.self.cycles-pp.f2fs_convert_inline_page
      0.03 ±103%      +0.1        0.11 ± 20%  perf-profile.self.cycles-pp.folio_account_dirtied
      0.03 ±105%      +0.1        0.11 ± 17%  perf-profile.self.cycles-pp.__write_node_page
      0.03 ±103%      +0.1        0.12 ± 15%  perf-profile.self.cycles-pp.__attach_extent_node
      0.10 ± 25%      +0.1        0.18 ± 18%  perf-profile.self.cycles-pp.__folio_end_writeback
      0.10 ± 21%      +0.1        0.18 ± 10%  perf-profile.self.cycles-pp.f2fs_submit_page_write
      0.16 ± 10%      +0.1        0.25 ± 22%  perf-profile.self.cycles-pp.__is_cp_guaranteed
      0.04 ± 77%      +0.1        0.13 ± 25%  perf-profile.self.cycles-pp.f2fs_do_sync_file
      0.15 ± 21%      +0.1        0.24 ± 12%  perf-profile.self.cycles-pp.rwsem_optimistic_spin
      0.02 ±149%      +0.1        0.11 ± 25%  perf-profile.self.cycles-pp.__submit_merged_write_cond
      0.11 ± 10%      +0.1        0.20 ± 11%  perf-profile.self.cycles-pp.f2fs_write_end_io
      0.02 ±149%      +0.1        0.13 ±  9%  perf-profile.self.cycles-pp.f2fs_is_valid_blkaddr
      0.02 ±223%      +0.1        0.12 ± 23%  perf-profile.self.cycles-pp.f2fs_get_node_page
      0.08 ± 37%      +0.1        0.19 ± 15%  perf-profile.self.cycles-pp.update_segment_mtime
      0.11 ± 22%      +0.1        0.23 ± 21%  perf-profile.self.cycles-pp.percpu_counter_add_batch
      0.10 ± 43%      +0.1        0.22 ± 15%  perf-profile.self.cycles-pp.osq_lock
      0.02 ±223%      +0.1        0.14 ± 21%  perf-profile.self.cycles-pp.f2fs_balance_fs
      0.23 ± 20%      +0.1        0.37 ±  5%  perf-profile.self.cycles-pp.mutex_lock
      0.09 ± 42%      +0.1        0.24 ± 12%  perf-profile.self.cycles-pp.f2fs_get_node_info
      0.29 ±  9%      +0.2        0.45 ±  7%  perf-profile.self.cycles-pp.up_read
      0.55 ±  7%      +0.2        0.74 ±  9%  perf-profile.self.cycles-pp.xas_load
      0.04 ±125%      +0.2        0.22 ± 16%  perf-profile.self.cycles-pp.f2fs_inode_chksum_verify
      0.16 ± 42%      +0.2        0.35 ±  9%  perf-profile.self.cycles-pp.rwsem_spin_on_owner
      0.17 ± 38%      +0.2        0.37 ± 13%  perf-profile.self.cycles-pp.update_sit_entry
      0.09 ± 57%      +0.2        0.30 ± 15%  perf-profile.self.cycles-pp.has_not_enough_free_secs
      0.28 ± 25%      +0.2        0.50 ± 13%  perf-profile.self.cycles-pp._raw_spin_trylock
      0.27 ± 21%      +0.3        0.52 ±  5%  perf-profile.self.cycles-pp.__radix_tree_lookup
      0.25 ± 22%      +0.3        0.50 ±  9%  perf-profile.self.cycles-pp.down_write
      0.11 ± 46%      +0.3        0.39 ± 16%  perf-profile.self.cycles-pp.f2fs_allocate_data_block
      0.12 ± 55%      +0.3        0.40 ± 22%  perf-profile.self.cycles-pp.__get_node_page
      0.78 ±  9%      +0.3        1.11 ±  4%  perf-profile.self.cycles-pp._raw_spin_lock_irqsave
      0.53 ± 14%      +0.4        0.94 ±  7%  perf-profile.self.cycles-pp.down_read
      1.41 ±  7%      +0.5        1.92 ±  5%  perf-profile.self.cycles-pp._raw_spin_lock
      0.71 ± 43%      +1.0        1.68 ± 35%  perf-profile.self.cycles-pp.timekeeping_max_deferment




Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


-- 
0-DAY CI Kernel Test Service
https://01.org/lkp



View attachment "config-5.18.0-rc5-00021-gbb2dee337bd7" of type "text/plain" (162679 bytes)

View attachment "job-script" of type "text/plain" (8043 bytes)

View attachment "job.yaml" of type "text/plain" (5386 bytes)

View attachment "reproduce" of type "text/plain" (277 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ