lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [day] [month] [year] [list]
Date:   Wed, 22 Jun 2022 13:55:59 +0800
From:   kernel test robot <oliver.sang@...el.com>
To:     Mel Gorman <mgorman@...hsingularity.net>
Cc:     Peter Zijlstra <peterz@...radead.org>,
        K Prateek Nayak <kprateek.nayak@....com>,
        LKML <linux-kernel@...r.kernel.org>, x86@...nel.org,
        lkp@...ts.01.org, lkp@...el.com, ying.huang@...el.com,
        feng.tang@...el.com, zhengjun.xing@...ux.intel.com,
        fengwei.yin@...el.com, guobing.chen@...el.com,
        ming.a.chen@...el.com, frank.du@...el.com, Shuhua.Fan@...el.com,
        wangyang.guo@...el.com, Wenhuan.Huang@...el.com,
        jessica.ji@...el.com, shan.kang@...el.com, guangli.li@...el.com,
        tiejun.li@...el.com, yu.ma@...el.com, dapeng1.mi@...el.com,
        jiebin.sun@...el.com, gengxin.xie@...el.com, fan.zhao@...el.com,
        aubrey.li@...ux.intel.com, yu.c.chen@...el.com
Subject: [sched/numa]  cb29a5c19d:
 phoronix-test-suite.vpxenc.Speed5.frames_per_second -8.0% regression


Hi, Mel Gorman,

we reported "[sched/numa]  bb2dee337b:  unixbench.score -11.2% regression"
last month at
https://lore.kernel.org/all/20220518092414.GA15472@xsang-OptiPlex-9020/
and according to your inputs and experiments from Ying, we got the idea this
commit could benefit some workloads but cause regression on others.

now we noticed this commit in tip now, and we found another regression in one
PTS test (the fsmark regression reported in bb2dee337b report is also
reproduced). just FYI


Greeting,

FYI, we noticed a -8.0% regression of phoronix-test-suite.vpxenc.Speed5.frames_per_second due to commit:


commit: cb29a5c19d2d68afc641fb1949e1a1c565b582ea ("sched/numa: Apply imbalance limitations consistently")
https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git sched/core

in testcase: phoronix-test-suite
on test machine: 96 threads 2 sockets Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 512G memory
with following parameters:

	test: vpxenc-3.0.0
	option_a: Speed 5 [Fastest - Default]
	cpufreq_governor: performance
	ucode: 0x500320a

test-description: The Phoronix Test Suite is the most comprehensive testing and benchmarking platform available that provides an extensible framework for which new tests can be easily added.
test-url: http://www.phoronix-test-suite.com/

In addition to that, the commit also has significant impact on the following tests:

+------------------+-------------------------------------------------------------------------------------+
| testcase: change | fsmark: fsmark.files_per_sec -13.8% regression                                      |
| test machine     | 192 threads 4 sockets Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory |
| test parameters  | cpufreq_governor=performance                                                        |
|                  | disk=1SSD                                                                           |
|                  | filesize=9B                                                                         |
|                  | fs=btrfs                                                                            |
|                  | iterations=8                                                                        |
|                  | nr_directories=16d                                                                  |
|                  | nr_files_per_directory=256fpd                                                       |
|                  | nr_threads=4                                                                        |
|                  | sync_method=fsyncBeforeClose                                                        |
|                  | test_size=16G                                                                       |
|                  | ucode=0x500320a                                                                     |
+------------------+-------------------------------------------------------------------------------------+


If you fix the issue, kindly add following tag
Reported-by: kernel test robot <oliver.sang@...el.com>


Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        sudo bin/lkp install job.yaml           # job file is attached in this email
        bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run
        sudo bin/lkp run generated-yaml-file

        # if come across any failure that blocks the test,
        # please remove ~/.lkp and /lkp dir to run from a clean state.

=========================================================================================
compiler/cpufreq_governor/kconfig/option_a/rootfs/tbox_group/test/testcase/ucode:
  gcc-11/performance/x86_64-rhel-8.3/Speed 5 [Fastest - Default]/debian-x86_64-phoronix/lkp-csl-2sp7/vpxenc-3.0.0/phoronix-test-suite/0x500320a

commit: 
  13ede33150 ("sched/numa: Do not swap tasks between nodes when spare capacity is available")
  cb29a5c19d ("sched/numa: Apply imbalance limitations consistently")

13ede33150877d44 cb29a5c19d2d68afc641fb1949e 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
    139.88            +7.1%     149.87        phoronix-test-suite.time.elapsed_time
    139.88            +7.1%     149.87        phoronix-test-suite.time.elapsed_time.max
    687138           -14.5%     587750 ±  5%  phoronix-test-suite.time.minor_page_faults
    350.44            +5.1%     368.43        phoronix-test-suite.time.user_time
    549751           +97.1%    1083494 ± 13%  phoronix-test-suite.time.voluntary_context_switches
     16.12            -8.0%      14.83        phoronix-test-suite.vpxenc.Speed5.frames_per_second
    257.88            +5.9%     273.13        pmeter.Average_Active_Power
      1.43 ± 18%      -0.4        1.05 ± 11%  mpstat.cpu.all.irq%
      0.07 ± 20%      -0.0        0.05 ± 12%  mpstat.cpu.all.soft%
     13365            -6.7%      12465        vmstat.io.bi
      9186           +69.7%      15590 ± 11%  vmstat.system.cs
    904355 ±  4%     -35.7%     581776 ± 28%  numa-numastat.node0.local_node
    939756 ±  3%     -34.3%     616963 ± 26%  numa-numastat.node0.numa_hit
    301216 ± 13%    +127.0%     683764 ± 24%  numa-numastat.node1.local_node
    334322 ± 10%    +114.4%     716912 ± 23%  numa-numastat.node1.numa_hit
      1957 ±  5%     +13.0%       2212 ±  4%  turbostat.Bzy_MHz
    456458 ± 36%    +279.2%    1731028 ± 85%  turbostat.C1
    139875 ± 11%     -40.7%      83002 ± 32%  turbostat.POLL
    132.56            +9.2%     144.73        turbostat.PkgWatt
    401285 ±  2%     -19.1%     324795 ±  8%  proc-vmstat.numa_hint_faults
    401245 ±  2%     -35.5%     258744 ± 11%  proc-vmstat.numa_hint_faults_local
   1280393            +4.7%    1340600        proc-vmstat.numa_hit
      2195 ±  2%     -20.9%       1737 ±  9%  proc-vmstat.numa_huge_pte_updates
   1211885            +5.0%    1272264        proc-vmstat.numa_local
    209.67 ±223%  +77390.1%     162471 ± 14%  proc-vmstat.numa_pages_migrated
   1553858           -20.1%    1242134 ±  9%  proc-vmstat.numa_pte_updates
   1285673            +4.5%    1344022        proc-vmstat.pgalloc_normal
    841292            +7.2%     901606        proc-vmstat.pgfree
    209.67 ±223%  +77390.1%     162471 ± 14%  proc-vmstat.pgmigrate_success
    124876            +4.4%     130409        proc-vmstat.pgreuse
    937171           -49.9%     469141 ± 67%  numa-meminfo.node0.Active
    931124           -50.3%     463046 ± 68%  numa-meminfo.node0.Active(file)
    325304           -51.6%     157469 ± 12%  numa-meminfo.node0.AnonHugePages
    584407           -50.4%     289749 ± 12%  numa-meminfo.node0.AnonPages
    651845           -38.4%     401650 ± 14%  numa-meminfo.node0.AnonPages.max
      4571 ±  3%     -17.6%       3766 ±  9%  numa-meminfo.node0.PageTables
     25612 ± 70%   +1863.0%     502765 ± 62%  numa-meminfo.node1.Active
     22402 ± 78%   +2130.4%     499671 ± 63%  numa-meminfo.node1.Active(file)
      3895 ± 25%   +4533.5%     180514 ± 13%  numa-meminfo.node1.AnonHugePages
     30561 ± 17%    +981.9%     330644 ±  9%  numa-meminfo.node1.AnonPages
     39832 ± 31%   +1131.7%     490599 ±  7%  numa-meminfo.node1.AnonPages.max
    166333 ±190%    +346.7%     742978 ± 59%  numa-meminfo.node1.Inactive(file)
   1192724 ± 42%    +108.8%    2490726 ± 33%  numa-meminfo.node1.MemUsed
      2059 ±  7%     +38.4%       2849 ± 12%  numa-meminfo.node1.PageTables
    232829           -50.3%     115776 ± 68%  numa-vmstat.node0.nr_active_file
    146027           -50.4%      72440 ± 12%  numa-vmstat.node0.nr_anon_pages
    158.33           -51.7%      76.50 ± 13%  numa-vmstat.node0.nr_anon_transparent_hugepages
      1142 ±  3%     -17.6%     941.33 ±  9%  numa-vmstat.node0.nr_page_table_pages
    232829           -50.3%     115776 ± 68%  numa-vmstat.node0.nr_zone_active_file
    939744 ±  3%     -34.4%     616740 ± 26%  numa-vmstat.node0.numa_hit
    904344 ±  4%     -35.7%     581554 ± 28%  numa-vmstat.node0.numa_local
      5600 ± 78%   +2126.1%     124669 ± 63%  numa-vmstat.node1.nr_active_file
      7640 ± 17%    +982.8%      82731 ±  9%  numa-vmstat.node1.nr_anon_pages
      1.33 ± 35%   +6475.0%      87.67 ± 13%  numa-vmstat.node1.nr_anon_transparent_hugepages
     41583 ±190%    +347.0%     185880 ± 59%  numa-vmstat.node1.nr_inactive_file
    515.00 ±  7%     +38.3%     712.50 ± 12%  numa-vmstat.node1.nr_page_table_pages
      5600 ± 78%   +2126.1%     124669 ± 63%  numa-vmstat.node1.nr_zone_active_file
     41583 ±190%    +347.0%     185880 ± 59%  numa-vmstat.node1.nr_zone_inactive_file
    334298 ± 10%    +114.5%     717103 ± 23%  numa-vmstat.node1.numa_hit
    301191 ± 13%    +127.1%     683955 ± 24%  numa-vmstat.node1.numa_local
 1.088e+09            -4.6%  1.037e+09        perf-stat.i.branch-instructions
      8.85 ± 15%      +7.7       16.53 ±  6%  perf-stat.i.cache-miss-rate%
   6329981 ±  9%     +61.9%   10246918 ±  4%  perf-stat.i.cache-misses
      9280           +71.2%      15887 ± 11%  perf-stat.i.context-switches
      2.70 ± 28%    +104.5%       5.51 ± 19%  perf-stat.i.cpu-migrations
      1857 ± 12%     -30.1%       1298 ± 18%  perf-stat.i.cycles-between-cache-misses
 1.519e+10            -5.9%  1.429e+10        perf-stat.i.instructions
     13279 ±  7%     -12.6%      11603 ±  3%  perf-stat.i.instructions-per-iTLB-miss
     12.53 ±  4%     -10.7%      11.18        perf-stat.i.major-faults
      8576           -11.7%       7571 ±  3%  perf-stat.i.minor-faults
     18.61 ± 13%     +52.9       71.50 ±  2%  perf-stat.i.node-load-miss-rate%
    148299 ± 27%   +1647.6%    2591752 ±  5%  perf-stat.i.node-load-misses
   1255857 ±  2%     -35.2%     814130 ±  9%  perf-stat.i.node-loads
     13.10 ± 14%     +50.3       63.44 ±  4%  perf-stat.i.node-store-miss-rate%
     42020 ± 17%   +1835.3%     813224 ±  6%  perf-stat.i.node-store-misses
    772924 ±  3%     -49.4%     391419 ±  8%  perf-stat.i.node-stores
      8589           -11.7%       7583 ±  3%  perf-stat.i.page-faults
      1867 ±103%    +222.2%       6016 ± 89%  sched_debug.cfs_rq:/.load_avg.max
    155431 ± 16%     -32.6%     104751 ± 14%  sched_debug.cfs_rq:/.min_vruntime.max
     24303 ± 19%     -37.6%      15171 ± 16%  sched_debug.cfs_rq:/.min_vruntime.stddev
    104.41 ±  6%     -14.8%      88.92 ±  7%  sched_debug.cfs_rq:/.runnable_avg.avg
    200.61 ±  3%     -11.9%     176.79 ±  4%  sched_debug.cfs_rq:/.runnable_avg.stddev
      8638 ± 90%    -358.6%     -22341        sched_debug.cfs_rq:/.spread0.avg
    107100 ± 21%     -70.1%      32044 ± 42%  sched_debug.cfs_rq:/.spread0.max
    -20872          +120.5%     -46014        sched_debug.cfs_rq:/.spread0.min
     24303 ± 19%     -37.6%      15174 ± 16%  sched_debug.cfs_rq:/.spread0.stddev
    103.49 ±  6%     -14.7%      88.31 ±  7%  sched_debug.cfs_rq:/.util_avg.avg
    198.85 ±  3%     -12.1%     174.80 ±  4%  sched_debug.cfs_rq:/.util_avg.stddev
     37.33 ± 18%     -39.3%      22.66 ± 19%  sched_debug.cfs_rq:/.util_est_enqueued.avg
    704.67 ± 10%     -22.2%     548.33 ± 14%  sched_debug.cfs_rq:/.util_est_enqueued.max
    131.21 ± 11%     -33.0%      87.86 ± 11%  sched_debug.cfs_rq:/.util_est_enqueued.stddev
     79565 ± 32%     +98.9%     158227 ± 39%  sched_debug.cpu.avg_idle.min
    341.80 ± 39%     -62.7%     127.44 ± 27%  sched_debug.cpu.clock_task.stddev
    245.03 ± 15%     -21.4%     192.50 ±  9%  sched_debug.cpu.curr->pid.avg
      1016 ±  8%     -15.3%     860.84 ±  5%  sched_debug.cpu.curr->pid.stddev
      7305           +48.3%      10834 ± 10%  sched_debug.cpu.nr_switches.avg
     99341 ± 16%     +79.6%     178446 ± 25%  sched_debug.cpu.nr_switches.max
     17397 ± 10%     +63.1%      28380 ± 18%  sched_debug.cpu.nr_switches.stddev
      0.18 ±142%      +0.6        0.80 ± 25%  perf-profile.calltrace.cycles-pp.tick_nohz_irq_exit.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
      0.54 ± 46%      +0.8        1.38 ± 33%  perf-profile.calltrace.cycles-pp.timekeeping_max_deferment.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.cpuidle_idle_call
      1.15 ± 17%      +1.0        2.16 ± 26%  perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.cpuidle_idle_call.do_idle
      1.46 ± 21%      +1.1        2.52 ± 24%  perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.cpuidle_idle_call.do_idle.cpu_startup_entry
      0.73 ± 27%      -0.2        0.52 ± 17%  perf-profile.children.cycles-pp.perf_trace_sched_stat_runtime
      0.53 ± 31%      -0.2        0.34 ± 11%  perf-profile.children.cycles-pp.task_tick_fair
      0.02 ±142%      +0.1        0.09 ± 34%  perf-profile.children.cycles-pp.flush_smp_call_function_queue
      0.04 ±104%      +0.1        0.15 ± 36%  perf-profile.children.cycles-pp.do_anonymous_page
      0.04 ±104%      +0.1        0.17 ± 30%  perf-profile.children.cycles-pp.clear_page_erms
      0.23 ± 26%      +0.1        0.37 ± 13%  perf-profile.children.cycles-pp.arch_cpu_idle_enter
      0.22 ± 22%      +0.1        0.36 ± 14%  perf-profile.children.cycles-pp.tsc_verify_tsc_adjust
      0.04 ±101%      +0.1        0.18 ± 29%  perf-profile.children.cycles-pp.__do_huge_pmd_anonymous_page
      0.04 ±101%      +0.1        0.18 ± 29%  perf-profile.children.cycles-pp.clear_huge_page
      0.29 ± 28%      +0.2        0.46 ± 33%  perf-profile.children.cycles-pp.ret_from_fork
      0.28 ± 28%      +0.2        0.46 ± 34%  perf-profile.children.cycles-pp.kthread
      0.19 ± 26%      +0.2        0.42 ± 14%  perf-profile.children.cycles-pp.calc_global_load_tick
      0.39 ± 26%      +0.2        0.63 ± 13%  perf-profile.children.cycles-pp.arch_scale_freq_tick
      0.60 ± 11%      +0.3        0.89 ± 15%  perf-profile.children.cycles-pp.asm_exc_page_fault
      0.35 ± 13%      +0.3        0.66 ± 15%  perf-profile.children.cycles-pp.do_user_addr_fault
      0.39 ± 11%      +0.3        0.70 ± 16%  perf-profile.children.cycles-pp.exc_page_fault
      0.30 ± 16%      +0.3        0.61 ± 17%  perf-profile.children.cycles-pp.handle_mm_fault
      0.27 ± 19%      +0.3        0.59 ± 17%  perf-profile.children.cycles-pp.__handle_mm_fault
      0.45 ± 19%      +0.4        0.84 ± 24%  perf-profile.children.cycles-pp.tick_nohz_irq_exit
      0.63 ± 17%      +0.8        1.38 ± 33%  perf-profile.children.cycles-pp.timekeeping_max_deferment
      1.18 ± 17%      +1.0        2.20 ± 26%  perf-profile.children.cycles-pp.tick_nohz_next_event
      1.48 ± 20%      +1.1        2.55 ± 23%  perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
      0.04 ±107%      +0.1        0.17 ± 31%  perf-profile.self.cycles-pp.clear_page_erms
      0.20 ± 20%      +0.1        0.34 ± 19%  perf-profile.self.cycles-pp.tsc_verify_tsc_adjust
      0.19 ± 25%      +0.2        0.41 ± 16%  perf-profile.self.cycles-pp.calc_global_load_tick
      0.39 ± 26%      +0.2        0.63 ± 13%  perf-profile.self.cycles-pp.arch_scale_freq_tick
      0.16 ± 19%      +0.2        0.40 ± 35%  perf-profile.self.cycles-pp.tick_sched_do_timer
      0.62 ± 17%      +0.8        1.38 ± 33%  perf-profile.self.cycles-pp.timekeeping_max_deferment


***************************************************************************************************
lkp-csl-2ap4: 192 threads 4 sockets Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
=========================================================================================
compiler/cpufreq_governor/disk/filesize/fs/iterations/kconfig/nr_directories/nr_files_per_directory/nr_threads/rootfs/sync_method/tbox_group/test_size/testcase/ucode:
  gcc-11/performance/1SSD/9B/btrfs/8/x86_64-rhel-8.3/16d/256fpd/4/debian-11.1-x86_64-20220510.cgz/fsyncBeforeClose/lkp-csl-2ap4/16G/fsmark/0x500320a

commit: 
  13ede33150 ("sched/numa: Do not swap tasks between nodes when spare capacity is available")
  cb29a5c19d ("sched/numa: Apply imbalance limitations consistently")

13ede33150877d44 cb29a5c19d2d68afc641fb1949e 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
   2267984           +10.3%    2501971        fsmark.app_overhead
     12670           -13.8%      10919        fsmark.files_per_sec
    350.44           +18.9%     416.58        fsmark.time.elapsed_time
    350.44           +18.9%     416.58        fsmark.time.elapsed_time.max
  3.15e+08            +5.9%  3.335e+08        fsmark.time.file_system_outputs
    161.50           +19.1%     192.33        fsmark.time.percent_of_cpu_this_job_got
    552.36 ±  3%     +42.5%     787.14        fsmark.time.system_time
  14387779           +13.0%   16251439        fsmark.time.voluntary_context_switches
      2.71 ± 13%     -19.4%       2.19        iostat.cpu.system
     12275 ±  5%     +22.8%      15070 ±  5%  meminfo.Active(anon)
 6.525e+10           +19.6%  7.807e+10        cpuidle..time
 1.476e+08 ±  8%     +24.0%   1.83e+08 ±  2%  cpuidle..usage
    387.74           +17.1%     453.90        uptime.boot
     71567           +17.8%      84337        uptime.idle
    423821           -10.2%     380430        vmstat.io.bo
     88136            -4.7%      83981        vmstat.system.cs
      1.75 ± 18%      -0.7        1.09        mpstat.cpu.all.irq%
      0.12 ± 24%      -0.0        0.08        mpstat.cpu.all.soft%
      0.85 ±  2%      +0.2        1.02        mpstat.cpu.all.sys%
      0.05 ±  2%      -0.0        0.04 ±  2%  mpstat.cpu.all.usr%
      6212 ± 94%    +196.8%      18437 ± 49%  numa-vmstat.node2.nr_anon_pages
      6331 ± 92%    +193.8%      18600 ± 49%  numa-vmstat.node2.nr_inactive_anon
      6331 ± 92%    +193.8%      18600 ± 49%  numa-vmstat.node2.nr_zone_inactive_anon
      1969 ± 14%     +25.2%       2465 ± 15%  numa-vmstat.node3.nr_active_anon
      1969 ± 14%     +25.2%       2465 ± 15%  numa-vmstat.node3.nr_zone_active_anon
     17059 ±116%    +234.6%      57086 ± 54%  numa-meminfo.node2.AnonHugePages
     24850 ± 94%    +196.8%      73756 ± 50%  numa-meminfo.node2.AnonPages
     46859 ± 83%    +221.6%     150678 ± 19%  numa-meminfo.node2.AnonPages.max
     25324 ± 92%    +193.8%      74408 ± 49%  numa-meminfo.node2.Inactive(anon)
      7877 ± 14%     +25.2%       9864 ± 15%  numa-meminfo.node3.Active(anon)
     23395 ± 95%    +153.4%      59277 ± 45%  numa-meminfo.node3.AnonHugePages
      4.41 ± 19%      -1.1        3.27        turbostat.Busy%
      1673 ±  7%     +32.3%       2214        turbostat.Bzy_MHz
   7354409 ±  5%     +26.5%    9300908 ±  2%  turbostat.C1
     11.52 ± 99%     -99.8%       0.03 ± 37%  turbostat.CPU%c6
  1.36e+08 ±  9%     +23.8%  1.683e+08        turbostat.IRQ
   5069307 ±  2%     +22.1%    6190906 ±  2%  turbostat.POLL
     93.82 ± 13%     +19.6%     112.16 ±  5%  turbostat.PkgWatt
      8.09 ± 33%     -45.9%       4.38 ± 53%  sched_debug.cfs_rq:/.removed.load_avg.avg
   1635183 ± 24%     -29.7%    1149085 ±  4%  sched_debug.cpu.avg_idle.max
    171308 ± 13%     +29.3%     221521 ±  4%  sched_debug.cpu.clock.avg
    171317 ± 13%     +29.3%     221530 ±  4%  sched_debug.cpu.clock.max
    171298 ± 13%     +29.3%     221511 ±  4%  sched_debug.cpu.clock.min
    168797 ± 13%     +29.9%     219332 ±  4%  sched_debug.cpu.clock_task.avg
    170371 ± 13%     +29.2%     220131 ±  4%  sched_debug.cpu.clock_task.max
    167274 ± 13%     +29.9%     217305 ±  4%  sched_debug.cpu.clock_task.min
    186.80 ± 11%     +26.5%     236.24 ±  7%  sched_debug.cpu.curr->pid.avg
     11025 ±  6%     +14.8%      12654 ±  2%  sched_debug.cpu.curr->pid.max
      1294 ± 10%     +23.1%       1593 ±  4%  sched_debug.cpu.curr->pid.stddev
    171297 ± 13%     +29.3%     221510 ±  4%  sched_debug.cpu_clk
    170281 ± 13%     +29.5%     220495 ±  4%  sched_debug.ktime
    173027 ± 13%     +29.0%     223193 ±  4%  sched_debug.sched_clk
      3069 ±  5%     +22.5%       3758 ±  4%  proc-vmstat.nr_active_anon
  39505514            +5.9%   41822347        proc-vmstat.nr_dirtied
     67438            -3.1%      65331        proc-vmstat.nr_dirty
      7699            -1.6%       7578        proc-vmstat.nr_mapped
     56.17 ± 12%    +148.1%     139.33        proc-vmstat.nr_mlock
      5984 ±  3%      +5.8%       6333 ±  4%  proc-vmstat.nr_shmem
    804117            +1.1%     813202        proc-vmstat.nr_slab_reclaimable
  39504067            +5.9%   41820963        proc-vmstat.nr_written
      3069 ±  5%     +22.5%       3758 ±  4%  proc-vmstat.nr_zone_active_anon
     67557            -3.1%      65462        proc-vmstat.nr_zone_write_pending
      3133 ± 80%    +546.1%      20248 ± 13%  proc-vmstat.numa_hint_faults
      1705 ± 98%    +491.9%      10092 ± 29%  proc-vmstat.numa_hint_faults_local
      4767 ±126%    +405.7%      24107 ± 22%  proc-vmstat.numa_pages_migrated
    280411           +33.0%     372905 ±  4%  proc-vmstat.numa_pte_updates
   4082927 ±  3%      -7.8%    3763069        proc-vmstat.pgactivate
   1214665           +15.3%    1400569        proc-vmstat.pgfault
      4767 ±126%    +405.7%      24107 ± 22%  proc-vmstat.pgmigrate_success
 1.498e+08            +6.6%  1.597e+08        proc-vmstat.pgpgout
     50215           +12.9%      56670        proc-vmstat.pgreuse
     18.49 ± 44%     -46.7%       9.86        perf-stat.i.MPKI
 1.327e+09 ±  2%      -8.5%  1.214e+09        perf-stat.i.branch-instructions
  23501825 ± 43%     -44.7%   12986923        perf-stat.i.branch-misses
     10.68 ± 30%     +26.1       36.81 ±  3%  perf-stat.i.cache-miss-rate%
  10372759 ± 24%    +102.9%   21042523 ±  2%  perf-stat.i.cache-misses
 1.196e+08 ± 43%     -50.4%   59271260        perf-stat.i.cache-references
     88799            -4.8%      84528        perf-stat.i.context-switches
      1382 ± 15%     -54.6%     627.98 ±  2%  perf-stat.i.cycles-between-cache-misses
 1.761e+09 ±  2%      -8.7%  1.608e+09        perf-stat.i.dTLB-loads
 8.631e+08 ±  2%      -8.8%  7.875e+08        perf-stat.i.dTLB-stores
   3220005 ±  4%      +7.9%    3474453 ±  3%  perf-stat.i.iTLB-load-misses
 6.638e+09 ±  2%      -8.7%  6.062e+09        perf-stat.i.instructions
      2077 ±  2%     -15.7%       1750 ±  3%  perf-stat.i.instructions-per-iTLB-miss
      1.35 ± 11%     -24.1%       1.02        perf-stat.i.major-faults
    666.06 ± 39%     -41.0%     392.91        perf-stat.i.metric.K/sec
     20.59 ±  2%      -8.7%      18.80        perf-stat.i.metric.M/sec
     34.66 ± 29%     +53.9       88.61        perf-stat.i.node-load-miss-rate%
   1031595 ± 62%    +589.3%    7111040 ±  2%  perf-stat.i.node-load-misses
   1777642 ±  6%     -50.1%     887280 ±  6%  perf-stat.i.node-loads
     40.72 ± 35%     +44.6       85.35        perf-stat.i.node-store-miss-rate%
    530303 ± 51%    +265.1%    1935875 ±  2%  perf-stat.i.node-store-misses
    729861 ± 18%     -55.2%     327192 ± 11%  perf-stat.i.node-stores
     18.12 ± 44%     -46.0%       9.78        perf-stat.overall.MPKI
      9.67 ± 25%     +25.8       35.51 ±  3%  perf-stat.overall.cache-miss-rate%
      1249 ± 13%     -51.3%     608.83        perf-stat.overall.cycles-between-cache-misses
      2063 ±  2%     -15.3%       1747 ±  3%  perf-stat.overall.instructions-per-iTLB-miss
     34.25 ± 38%     +54.6       88.89        perf-stat.overall.node-load-miss-rate%
     40.46 ± 35%     +45.1       85.54        perf-stat.overall.node-store-miss-rate%
 1.323e+09 ±  2%      -8.5%  1.211e+09        perf-stat.ps.branch-instructions
  23433121 ± 43%     -44.7%   12956511        perf-stat.ps.branch-misses
  10344317 ± 24%    +102.9%   20990278 ±  2%  perf-stat.ps.cache-misses
 1.193e+08 ± 43%     -50.4%   59122669        perf-stat.ps.cache-references
     88533            -4.8%      84309        perf-stat.ps.context-switches
 1.756e+09 ±  2%      -8.6%  1.605e+09        perf-stat.ps.dTLB-loads
 8.606e+08 ±  2%      -8.7%  7.856e+08        perf-stat.ps.dTLB-stores
   3210622 ±  4%      +8.0%    3465877 ±  3%  perf-stat.ps.iTLB-load-misses
 6.619e+09 ±  2%      -8.6%  6.048e+09        perf-stat.ps.instructions
      1.35 ± 11%     -24.1%       1.02        perf-stat.ps.major-faults
   1028434 ± 62%    +589.7%    7092993 ±  2%  perf-stat.ps.node-load-misses
   1773152 ±  6%     -50.1%     885261 ±  6%  perf-stat.ps.node-loads
    528782 ± 51%    +265.2%    1931070 ±  2%  perf-stat.ps.node-store-misses
    727924 ± 18%     -55.2%     326470 ± 11%  perf-stat.ps.node-stores
 2.326e+12            +8.6%  2.525e+12        perf-stat.total.instructions
     59.09 ±  4%      -5.1       54.02 ±  2%  perf-profile.calltrace.cycles-pp.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
     58.31 ±  4%      -4.9       53.46 ±  2%  perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry
     63.06 ±  4%      -4.5       58.54 ±  2%  perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
     64.98 ±  4%      -4.3       60.72 ±  2%  perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
     64.76 ±  4%      -4.2       60.56 ±  2%  perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
     64.76 ±  4%      -4.2       60.56 ±  2%  perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
     64.72 ±  4%      -4.2       60.53 ±  2%  perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
      4.53 ± 10%      -1.3        3.20 ± 10%  perf-profile.calltrace.cycles-pp.wait_log_commit.btrfs_sync_log.btrfs_sync_file.__x64_sys_fsync.do_syscall_64
      3.57 ± 10%      -1.2        2.40 ± 10%  perf-profile.calltrace.cycles-pp.__mutex_lock.wait_log_commit.btrfs_sync_log.btrfs_sync_file.__x64_sys_fsync
      2.63 ± 12%      -0.9        1.69 ±  9%  perf-profile.calltrace.cycles-pp.mutex_spin_on_owner.__mutex_lock.wait_log_commit.btrfs_sync_log.btrfs_sync_file
      3.93 ±  8%      -0.6        3.34 ±  8%  perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
      0.92 ± 10%      -0.2        0.68 ± 15%  perf-profile.calltrace.cycles-pp.osq_lock.__mutex_lock.wait_log_commit.btrfs_sync_log.btrfs_sync_file
      1.16 ±  9%      -0.2        0.95 ± 12%  perf-profile.calltrace.cycles-pp.__btrfs_tree_lock.btrfs_search_slot.btrfs_insert_empty_items.btrfs_create_new_inode.btrfs_create_common
      1.13 ±  9%      -0.2        0.94 ± 12%  perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.__btrfs_tree_lock.btrfs_search_slot.btrfs_insert_empty_items.btrfs_create_new_inode
      0.82 ±  7%      -0.2        0.64 ± 11%  perf-profile.calltrace.cycles-pp.__schedule.schedule.wait_log_commit.btrfs_sync_log.btrfs_sync_file
      0.82 ±  7%      -0.2        0.64 ± 12%  perf-profile.calltrace.cycles-pp.schedule.wait_log_commit.btrfs_sync_log.btrfs_sync_file.__x64_sys_fsync
      1.56 ±  6%      +0.2        1.76 ±  5%  perf-profile.calltrace.cycles-pp.write
      1.55 ±  6%      +0.2        1.75 ±  5%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write
      1.54 ±  6%      +0.2        1.74 ±  5%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
      1.45 ±  6%      +0.2        1.65 ±  6%  perf-profile.calltrace.cycles-pp.btrfs_buffered_write.btrfs_do_write_iter.new_sync_write.vfs_write.ksys_write
      1.52 ±  6%      +0.2        1.73 ±  5%  perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
      1.53 ±  6%      +0.2        1.74 ±  5%  perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
      1.50 ±  6%      +0.2        1.70 ±  5%  perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.47 ±  7%      +0.2        1.68 ±  6%  perf-profile.calltrace.cycles-pp.btrfs_do_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64
      0.90 ± 13%      +0.2        1.12 ±  5%  perf-profile.calltrace.cycles-pp.btrfs_lookup_file_extent.btrfs_drop_extents.cow_file_range_inline.cow_file_range.btrfs_run_delalloc_range
      0.90 ± 13%      +0.2        1.11 ±  5%  perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_lookup_file_extent.btrfs_drop_extents.cow_file_range_inline.cow_file_range
      1.30 ± 13%      +0.3        1.57 ±  5%  perf-profile.calltrace.cycles-pp.btrfs_drop_extents.cow_file_range_inline.cow_file_range.btrfs_run_delalloc_range.writepage_delalloc
      0.64 ± 14%      +0.3        0.93 ±  6%  perf-profile.calltrace.cycles-pp.btrfs_alloc_tree_block.__btrfs_cow_block.btrfs_cow_block.btrfs_search_slot.btrfs_insert_empty_items
      1.98 ± 12%      +0.3        2.32 ±  5%  perf-profile.calltrace.cycles-pp.cow_file_range_inline.cow_file_range.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage
      2.58 ± 11%      +0.4        2.94 ±  5%  perf-profile.calltrace.cycles-pp.start_ordered_ops.btrfs_sync_file.__x64_sys_fsync.do_syscall_64.entry_SYSCALL_64_after_hwframe
      2.56 ± 11%      +0.4        2.93 ±  5%  perf-profile.calltrace.cycles-pp.__filemap_fdatawrite_range.start_ordered_ops.btrfs_sync_file.__x64_sys_fsync.do_syscall_64
      2.51 ± 11%      +0.4        2.88 ±  5%  perf-profile.calltrace.cycles-pp.extent_write_cache_pages.extent_writepages.do_writepages.filemap_fdatawrite_wbc.__filemap_fdatawrite_range
      2.53 ± 11%      +0.4        2.90 ±  5%  perf-profile.calltrace.cycles-pp.do_writepages.filemap_fdatawrite_wbc.__filemap_fdatawrite_range.start_ordered_ops.btrfs_sync_file
      2.52 ± 11%      +0.4        2.90 ±  5%  perf-profile.calltrace.cycles-pp.extent_writepages.do_writepages.filemap_fdatawrite_wbc.__filemap_fdatawrite_range.start_ordered_ops
      2.55 ± 11%      +0.4        2.92 ±  5%  perf-profile.calltrace.cycles-pp.filemap_fdatawrite_wbc.__filemap_fdatawrite_range.start_ordered_ops.btrfs_sync_file.__x64_sys_fsync
      2.34 ± 12%      +0.4        2.74 ±  4%  perf-profile.calltrace.cycles-pp.__extent_writepage.extent_write_cache_pages.extent_writepages.do_writepages.filemap_fdatawrite_wbc
      2.25 ± 12%      +0.4        2.66 ±  5%  perf-profile.calltrace.cycles-pp.cow_file_range.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage.extent_write_cache_pages
      2.25 ± 12%      +0.4        2.67 ±  5%  perf-profile.calltrace.cycles-pp.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage.extent_write_cache_pages.extent_writepages
      0.77 ± 10%      +0.5        1.23 ±  4%  perf-profile.calltrace.cycles-pp.__btrfs_cow_block.btrfs_cow_block.btrfs_search_slot.btrfs_update_root.btrfs_sync_log
      0.78 ± 10%      +0.5        1.24 ±  5%  perf-profile.calltrace.cycles-pp.btrfs_cow_block.btrfs_search_slot.btrfs_update_root.btrfs_sync_log.btrfs_sync_file
      0.84 ±  9%      +0.5        1.35 ±  4%  perf-profile.calltrace.cycles-pp.btrfs_update_root.btrfs_sync_log.btrfs_sync_file.__x64_sys_fsync.do_syscall_64
      0.82 ± 11%      +0.5        1.33 ±  4%  perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_update_root.btrfs_sync_log.btrfs_sync_file.__x64_sys_fsync
      0.09 ±223%      +0.5        0.60 ±  9%  perf-profile.calltrace.cycles-pp.btrfs_init_new_buffer.btrfs_alloc_tree_block.__btrfs_cow_block.btrfs_cow_block.btrfs_search_slot
      0.88 ± 10%      +0.5        1.39 ± 10%  perf-profile.calltrace.cycles-pp.__mem_cgroup_flush_stats.mem_cgroup_wb_stats.balance_dirty_pages.balance_dirty_pages_ratelimited.btrfs_create_common
      0.87 ±  9%      +0.5        1.39 ± 10%  perf-profile.calltrace.cycles-pp.cgroup_rstat_flush_irqsafe.__mem_cgroup_flush_stats.mem_cgroup_wb_stats.balance_dirty_pages.balance_dirty_pages_ratelimited
      0.86 ±  9%      +0.5        1.38 ± 10%  perf-profile.calltrace.cycles-pp.cgroup_rstat_flush_locked.cgroup_rstat_flush_irqsafe.__mem_cgroup_flush_stats.mem_cgroup_wb_stats.balance_dirty_pages
      0.89 ± 10%      +0.5        1.41 ± 10%  perf-profile.calltrace.cycles-pp.mem_cgroup_wb_stats.balance_dirty_pages.balance_dirty_pages_ratelimited.btrfs_create_common.lookup_open
      0.90 ± 10%      +0.5        1.43 ± 10%  perf-profile.calltrace.cycles-pp.balance_dirty_pages.balance_dirty_pages_ratelimited.btrfs_create_common.lookup_open.open_last_lookups
      0.92 ±  9%      +0.5        1.46 ±  9%  perf-profile.calltrace.cycles-pp.balance_dirty_pages_ratelimited.btrfs_create_common.lookup_open.open_last_lookups.path_openat
      0.00            +0.6        0.62 ±  6%  perf-profile.calltrace.cycles-pp.btrfs_search_slot.inode_logged.btrfs_log_inode.btrfs_log_inode_parent.btrfs_log_dentry_safe
      0.00            +0.7        0.66 ±  6%  perf-profile.calltrace.cycles-pp.inode_logged.btrfs_log_inode.btrfs_log_inode_parent.btrfs_log_dentry_safe.btrfs_sync_file
      0.00            +0.7        0.69 ± 12%  perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.cgroup_rstat_flush_locked.cgroup_rstat_flush_irqsafe.__mem_cgroup_flush_stats.mem_cgroup_wb_stats
      0.92 ± 11%      +0.8        1.68 ± 11%  perf-profile.calltrace.cycles-pp.copy_extent_buffer_full.__btrfs_cow_block.btrfs_cow_block.btrfs_search_slot.btrfs_insert_empty_items
      0.00            +0.8        0.76 ±  7%  perf-profile.calltrace.cycles-pp.osq_lock.__mutex_lock.start_log_trans.btrfs_log_inode_parent.btrfs_log_dentry_safe
      4.71 ±  7%      +0.8        5.52 ±  4%  perf-profile.calltrace.cycles-pp.btrfs_create_common.lookup_open.open_last_lookups.path_openat.do_filp_open
      5.99 ±  7%      +0.8        6.82 ±  4%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.open64
      5.97 ±  7%      +0.8        6.81 ±  4%  perf-profile.calltrace.cycles-pp.__x64_sys_openat.do_syscall_64.entry_SYSCALL_64_after_hwframe.open64
      6.00 ±  7%      +0.8        6.85 ±  4%  perf-profile.calltrace.cycles-pp.open64
      5.96 ±  7%      +0.8        6.80 ±  4%  perf-profile.calltrace.cycles-pp.do_sys_openat2.__x64_sys_openat.do_syscall_64.entry_SYSCALL_64_after_hwframe.open64
      5.90 ±  7%      +0.8        6.74 ±  4%  perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_openat2.__x64_sys_openat.do_syscall_64.entry_SYSCALL_64_after_hwframe
      5.98 ±  7%      +0.8        6.82 ±  4%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.open64
      5.89 ±  7%      +0.8        6.74 ±  4%  perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_openat2.__x64_sys_openat.do_syscall_64
      5.57 ±  6%      +0.9        6.44 ±  4%  perf-profile.calltrace.cycles-pp.lookup_open.open_last_lookups.path_openat.do_filp_open.do_sys_openat2
      5.63 ±  7%      +0.9        6.50 ±  4%  perf-profile.calltrace.cycles-pp.open_last_lookups.path_openat.do_filp_open.do_sys_openat2.__x64_sys_openat
      0.48 ±100%      +1.0        1.45 ± 25%  perf-profile.calltrace.cycles-pp.timekeeping_max_deferment.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.cpuidle_idle_call
      1.54 ± 28%      +1.1        2.64 ± 15%  perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.cpuidle_idle_call.do_idle.cpu_startup_entry
      0.80 ±  8%      +1.1        1.91 ±  5%  perf-profile.calltrace.cycles-pp.__mutex_lock.start_log_trans.btrfs_log_inode_parent.btrfs_log_dentry_safe.btrfs_sync_file
      0.00            +1.1        1.11 ±  6%  perf-profile.calltrace.cycles-pp.mutex_spin_on_owner.__mutex_lock.start_log_trans.btrfs_log_inode_parent.btrfs_log_dentry_safe
      0.84 ±  8%      +1.1        1.98 ±  4%  perf-profile.calltrace.cycles-pp.start_log_trans.btrfs_log_inode_parent.btrfs_log_dentry_safe.btrfs_sync_file.__x64_sys_fsync
      1.18 ± 39%      +1.2        2.34 ± 16%  perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.cpuidle_idle_call.do_idle
      0.91 ± 11%      +1.2        2.14 ± 18%  perf-profile.calltrace.cycles-pp.copy_page.copy_extent_buffer_full.__btrfs_cow_block.btrfs_cow_block.btrfs_search_slot
      2.22 ±  8%      +1.4        3.65 ±  6%  perf-profile.calltrace.cycles-pp.__btrfs_cow_block.btrfs_cow_block.btrfs_search_slot.btrfs_insert_empty_items.copy_items
      2.23 ±  8%      +1.4        3.67 ±  6%  perf-profile.calltrace.cycles-pp.btrfs_cow_block.btrfs_search_slot.btrfs_insert_empty_items.copy_items.copy_inode_items_to_log
      2.84 ±  7%      +2.2        5.01 ±  6%  perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_insert_empty_items.copy_items.copy_inode_items_to_log.btrfs_log_inode
      2.90 ±  7%      +2.2        5.07 ±  6%  perf-profile.calltrace.cycles-pp.btrfs_insert_empty_items.copy_items.copy_inode_items_to_log.btrfs_log_inode.btrfs_log_inode_parent
      3.07 ±  6%      +2.2        5.28 ±  6%  perf-profile.calltrace.cycles-pp.copy_items.copy_inode_items_to_log.btrfs_log_inode.btrfs_log_inode_parent.btrfs_log_dentry_safe
      4.44 ±  7%      +2.3        6.76 ±  5%  perf-profile.calltrace.cycles-pp.copy_inode_items_to_log.btrfs_log_inode.btrfs_log_inode_parent.btrfs_log_dentry_safe.btrfs_sync_file
      5.24 ±  7%      +2.8        8.02 ±  4%  perf-profile.calltrace.cycles-pp.btrfs_log_inode.btrfs_log_inode_parent.btrfs_log_dentry_safe.btrfs_sync_file.__x64_sys_fsync
     22.32 ±  8%      +3.4       25.74 ±  3%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.fsync
     22.40 ±  8%      +3.4       25.83 ±  3%  perf-profile.calltrace.cycles-pp.fsync
     22.31 ±  8%      +3.4       25.74 ±  3%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.fsync
     22.29 ±  8%      +3.4       25.72 ±  3%  perf-profile.calltrace.cycles-pp.__x64_sys_fsync.do_syscall_64.entry_SYSCALL_64_after_hwframe.fsync
     22.28 ±  8%      +3.4       25.71 ±  3%  perf-profile.calltrace.cycles-pp.btrfs_sync_file.__x64_sys_fsync.do_syscall_64.entry_SYSCALL_64_after_hwframe.fsync
      6.19 ±  7%      +4.0       10.18 ±  3%  perf-profile.calltrace.cycles-pp.btrfs_log_inode_parent.btrfs_log_dentry_safe.btrfs_sync_file.__x64_sys_fsync.do_syscall_64
      6.20 ±  7%      +4.0       10.20 ±  4%  perf-profile.calltrace.cycles-pp.btrfs_log_dentry_safe.btrfs_sync_file.__x64_sys_fsync.do_syscall_64.entry_SYSCALL_64_after_hwframe
     59.30 ±  4%      -5.1       54.15 ±  2%  perf-profile.children.cycles-pp.cpuidle_enter
     59.24 ±  4%      -5.1       54.10 ±  2%  perf-profile.children.cycles-pp.cpuidle_enter_state
     63.31 ±  4%      -4.6       58.71 ±  2%  perf-profile.children.cycles-pp.cpuidle_idle_call
     64.98 ±  4%      -4.3       60.71 ±  2%  perf-profile.children.cycles-pp.do_idle
     64.98 ±  4%      -4.3       60.72 ±  2%  perf-profile.children.cycles-pp.secondary_startup_64_no_verify
     64.98 ±  4%      -4.3       60.72 ±  2%  perf-profile.children.cycles-pp.cpu_startup_entry
     64.76 ±  4%      -4.2       60.56 ±  2%  perf-profile.children.cycles-pp.start_secondary
      4.53 ± 10%      -1.3        3.20 ± 10%  perf-profile.children.cycles-pp.wait_log_commit
      4.24 ±  7%      -0.6        3.68 ±  8%  perf-profile.children.cycles-pp.__hrtimer_run_queues
      0.60 ±  5%      -0.1        0.48 ± 15%  perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
      0.57 ±  6%      -0.1        0.46 ± 10%  perf-profile.children.cycles-pp.load_balance
      0.19 ± 41%      -0.1        0.10 ± 11%  perf-profile.children.cycles-pp.enqueue_hrtimer
      0.54 ±  8%      -0.1        0.46 ± 10%  perf-profile.children.cycles-pp.autoremove_wake_function
      0.26 ± 20%      -0.1        0.19 ±  9%  perf-profile.children.cycles-pp.__hrtimer_next_event_base
      0.44 ±  8%      -0.1        0.38 ± 11%  perf-profile.children.cycles-pp.find_busiest_group
      0.40 ±  8%      -0.1        0.34 ±  6%  perf-profile.children.cycles-pp.update_rq_clock
      0.27 ± 11%      -0.1        0.21 ± 12%  perf-profile.children.cycles-pp.get_next_timer_interrupt
      0.20 ± 13%      -0.1        0.14 ± 21%  perf-profile.children.cycles-pp.hrtimer_update_next_event
      0.31 ±  6%      -0.1        0.25 ± 13%  perf-profile.children.cycles-pp.pick_next_task_fair
      0.08 ± 23%      -0.1        0.03 ±100%  perf-profile.children.cycles-pp.perf_event_task_tick
      0.22 ±  7%      -0.0        0.18 ±  9%  perf-profile.children.cycles-pp.btrfs_comp_cpu_keys
      0.28 ±  5%      -0.0        0.25 ±  6%  perf-profile.children.cycles-pp.mkdir
      0.07 ± 11%      +0.0        0.09 ±  7%  perf-profile.children.cycles-pp.btrfs_delalloc_reserve_metadata
      0.07 ± 20%      +0.0        0.10 ±  6%  perf-profile.children.cycles-pp.rcu_do_batch
      0.10 ± 13%      +0.0        0.13 ±  8%  perf-profile.children.cycles-pp.select_task_rq
      0.20 ± 10%      +0.0        0.24 ±  5%  perf-profile.children.cycles-pp.kmem_cache_free
      0.03 ±100%      +0.0        0.07 ± 11%  perf-profile.children.cycles-pp.log_all_new_ancestors
      0.03 ±100%      +0.0        0.07 ± 15%  perf-profile.children.cycles-pp.btrfs_get_delayed_node
      0.10 ± 14%      +0.0        0.14 ±  9%  perf-profile.children.cycles-pp.up_read
      0.08 ± 23%      +0.0        0.12 ± 23%  perf-profile.children.cycles-pp.xas_start
      0.02 ±142%      +0.0        0.06 ± 14%  perf-profile.children.cycles-pp.btrfs_clean_tree_block
      0.18 ±  8%      +0.0        0.23 ±  4%  perf-profile.children.cycles-pp.btrfs_read_node_slot
      0.09 ± 12%      +0.0        0.14 ± 12%  perf-profile.children.cycles-pp.btrfs_use_block_rsv
      0.16 ± 13%      +0.0        0.21 ±  8%  perf-profile.children.cycles-pp.__process_pages_contig
      0.16 ± 13%      +0.0        0.21 ± 13%  perf-profile.children.cycles-pp.read_tree_block
      0.08 ± 18%      +0.0        0.12 ±  7%  perf-profile.children.cycles-pp.mark_page_accessed
      0.01 ±223%      +0.0        0.06 ± 11%  perf-profile.children.cycles-pp.__btrfs_add_delayed_item
      0.17 ± 16%      +0.1        0.22 ±  6%  perf-profile.children.cycles-pp.release_extent_buffer
      0.02 ±141%      +0.1        0.07 ± 16%  perf-profile.children.cycles-pp.xas_set_mark
      0.03 ±100%      +0.1        0.08 ±  8%  perf-profile.children.cycles-pp.crc32c_pcl_intel_digest
      0.10 ± 21%      +0.1        0.16 ± 10%  perf-profile.children.cycles-pp.percpu_counter_add_batch
      0.06 ± 23%      +0.1        0.12 ± 12%  perf-profile.children.cycles-pp.need_preemptive_reclaim
      0.00            +0.1        0.06 ±  8%  perf-profile.children.cycles-pp.sb_mark_inode_writeback
      0.23 ±  3%      +0.1        0.29 ±  6%  perf-profile.children.cycles-pp.pin_down_extent
      0.26 ± 10%      +0.1        0.32 ±  4%  perf-profile.children.cycles-pp.__folio_start_writeback
      0.03 ±100%      +0.1        0.09 ±  6%  perf-profile.children.cycles-pp.rwsem_mark_wake
      0.12 ± 16%      +0.1        0.18 ± 10%  perf-profile.children.cycles-pp.folio_account_dirtied
      0.06 ± 49%      +0.1        0.13 ± 10%  perf-profile.children.cycles-pp.cgroup_rstat_updated
      0.30 ±  8%      +0.1        0.37 ±  7%  perf-profile.children.cycles-pp.btrfs_unlock_up_safe
      0.01 ±223%      +0.1        0.08 ± 11%  perf-profile.children.cycles-pp.verify_parent_transid
      0.29 ±  7%      +0.1        0.36 ±  5%  perf-profile.children.cycles-pp.btrfs_free_tree_block
      0.05 ± 73%      +0.1        0.12 ± 12%  perf-profile.children.cycles-pp.btrfs_setup_item_for_insert
      0.20 ± 10%      +0.1        0.28 ± 11%  perf-profile.children.cycles-pp.find_free_extent
      0.00            +0.1        0.08 ± 19%  perf-profile.children.cycles-pp.llist_add_batch
      0.22 ± 12%      +0.1        0.30 ±  5%  perf-profile.children.cycles-pp.write_dev_supers
      0.00            +0.1        0.08 ± 14%  perf-profile.children.cycles-pp.__smp_call_single_queue
      0.25 ± 15%      +0.1        0.33 ± 21%  perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
      0.00            +0.1        0.09 ± 19%  perf-profile.children.cycles-pp.__flush_smp_call_function_queue
      0.11 ± 17%      +0.1        0.21 ± 11%  perf-profile.children.cycles-pp.__mod_memcg_lruvec_state
      0.17 ± 17%      +0.1        0.27 ±  6%  perf-profile.children.cycles-pp.btrfs_reserve_metadata_bytes
      0.26 ± 16%      +0.1        0.36 ±  8%  perf-profile.children.cycles-pp.unlock_up
      0.08 ± 21%      +0.1        0.18 ±  9%  perf-profile.children.cycles-pp.btrfs_buffer_uptodate
      0.17 ± 10%      +0.1        0.26 ± 10%  perf-profile.children.cycles-pp.btrfs_root_node
      0.23 ± 19%      +0.1        0.34 ± 14%  perf-profile.children.cycles-pp.start_transaction
      0.26 ± 12%      +0.1        0.38 ± 10%  perf-profile.children.cycles-pp.down_read
      0.21 ± 17%      +0.1        0.32 ±  7%  perf-profile.children.cycles-pp.mutex_lock
      0.18 ± 14%      +0.1        0.29 ±  6%  perf-profile.children.cycles-pp.__reserve_bytes
      0.60 ± 10%      +0.1        0.71 ±  7%  perf-profile.children.cycles-pp.xas_load
      0.36 ±  9%      +0.1        0.48 ±  4%  perf-profile.children.cycles-pp.btrfs_write_check
      0.35 ± 10%      +0.1        0.46 ±  3%  perf-profile.children.cycles-pp.file_remove_privs
      0.34 ± 11%      +0.1        0.46 ±  3%  perf-profile.children.cycles-pp.security_inode_need_killpriv
      0.34 ± 10%      +0.1        0.46 ±  3%  perf-profile.children.cycles-pp.cap_inode_need_killpriv
      0.34 ± 10%      +0.1        0.46 ±  3%  perf-profile.children.cycles-pp.__vfs_getxattr
      0.66 ±  9%      +0.1        0.78 ±  4%  perf-profile.children.cycles-pp.generic_bin_search
      0.26 ± 13%      +0.1        0.39 ±  3%  perf-profile.children.cycles-pp.btrfs_lookup_xattr
      0.03 ±100%      +0.1        0.16 ±  8%  perf-profile.children.cycles-pp.ttwu_queue_wakelist
      0.31 ± 12%      +0.1        0.44 ±  3%  perf-profile.children.cycles-pp.btrfs_getxattr
      0.19 ± 16%      +0.1        0.34 ± 22%  perf-profile.children.cycles-pp.tsc_verify_tsc_adjust
      0.20 ± 14%      +0.1        0.35 ± 23%  perf-profile.children.cycles-pp.arch_cpu_idle_enter
      0.34 ± 16%      +0.2        0.50 ±  3%  perf-profile.children.cycles-pp.btrfs_log_all_xattrs
      0.25 ± 17%      +0.2        0.40 ± 12%  perf-profile.children.cycles-pp.__folio_mark_dirty
      0.42 ± 10%      +0.2        0.58 ±  8%  perf-profile.children.cycles-pp.mem_cgroup_css_rstat_flush
      0.17 ± 14%      +0.2        0.32 ± 10%  perf-profile.children.cycles-pp.inode_tree_add
      0.12 ± 20%      +0.2        0.28 ± 11%  perf-profile.children.cycles-pp.btrfs_lock_root_node
      0.83 ±  8%      +0.2        1.01 ±  7%  perf-profile.children.cycles-pp.wake_up_q
      0.72 ±  9%      +0.2        0.91 ± 14%  perf-profile.children.cycles-pp.end_bio_extent_buffer_writepage
      0.02 ±144%      +0.2        0.21 ± 15%  perf-profile.children.cycles-pp.sched_ttwu_pending
      0.48 ± 12%      +0.2        0.67 ± 12%  perf-profile.children.cycles-pp.filemap_dirty_folio
      1.57 ±  6%      +0.2        1.77 ±  6%  perf-profile.children.cycles-pp.write
      1.45 ±  6%      +0.2        1.65 ±  6%  perf-profile.children.cycles-pp.btrfs_buffered_write
      0.74 ± 12%      +0.2        0.94 ±  4%  perf-profile.children.cycles-pp.find_extent_buffer_nolock
      0.37 ± 14%      +0.2        0.58 ±  7%  perf-profile.children.cycles-pp.set_extent_buffer_dirty
      1.47 ±  7%      +0.2        1.68 ±  6%  perf-profile.children.cycles-pp.btrfs_do_write_iter
      0.90 ± 13%      +0.2        1.12 ±  4%  perf-profile.children.cycles-pp.btrfs_lookup_file_extent
      0.40 ± 13%      +0.2        0.62 ±  7%  perf-profile.children.cycles-pp.btrfs_mark_buffer_dirty
      1.79 ±  8%      +0.2        2.02 ±  4%  perf-profile.children.cycles-pp.ksys_write
      1.78 ±  8%      +0.2        2.02 ±  4%  perf-profile.children.cycles-pp.vfs_write
      0.62 ± 11%      +0.2        0.86 ±  8%  perf-profile.children.cycles-pp.rwsem_spin_on_owner
      0.88 ±  9%      +0.2        1.11 ±  7%  perf-profile.children.cycles-pp.rwsem_wake
      1.75 ±  8%      +0.2        1.99 ±  4%  perf-profile.children.cycles-pp.new_sync_write
      0.97 ± 12%      +0.3        1.22 ±  4%  perf-profile.children.cycles-pp.find_extent_buffer
      1.30 ± 13%      +0.3        1.57 ±  5%  perf-profile.children.cycles-pp.btrfs_drop_extents
      0.57 ± 24%      +0.3        0.86 ± 15%  perf-profile.children.cycles-pp.btrfs_init_new_buffer
      0.36 ± 11%      +0.3        0.66 ±  6%  perf-profile.children.cycles-pp.inode_logged
      0.00            +0.3        0.30 ± 11%  perf-profile.children.cycles-pp.flush_smp_call_function_queue
      1.09 ± 12%      +0.3        1.41 ±  6%  perf-profile.children.cycles-pp.read_block_for_search
      0.95 ± 12%      +0.3        1.28 ±  5%  perf-profile.children.cycles-pp.rwsem_down_read_slowpath
      1.98 ± 12%      +0.3        2.32 ±  5%  perf-profile.children.cycles-pp.cow_file_range_inline
      2.58 ± 11%      +0.4        2.94 ±  5%  perf-profile.children.cycles-pp.start_ordered_ops
      2.52 ± 11%      +0.4        2.90 ±  5%  perf-profile.children.cycles-pp.extent_write_cache_pages
      2.55 ± 11%      +0.4        2.94 ±  4%  perf-profile.children.cycles-pp.extent_writepages
      2.34 ± 12%      +0.4        2.74 ±  4%  perf-profile.children.cycles-pp.__extent_writepage
      0.95 ± 16%      +0.4        1.35 ± 10%  perf-profile.children.cycles-pp.btrfs_alloc_tree_block
      2.25 ± 12%      +0.4        2.66 ±  5%  perf-profile.children.cycles-pp.cow_file_range
      2.26 ± 12%      +0.4        2.67 ±  5%  perf-profile.children.cycles-pp.btrfs_run_delalloc_range
      1.20 ± 10%      +0.4        1.62 ±  4%  perf-profile.children.cycles-pp.__btrfs_tree_read_lock
      0.64 ± 11%      +0.5        1.09 ±  3%  perf-profile.children.cycles-pp.btrfs_read_lock_root_node
      0.84 ±  9%      +0.5        1.35 ±  4%  perf-profile.children.cycles-pp.btrfs_update_root
      1.13 ±  5%      +0.5        1.64 ±  9%  perf-profile.children.cycles-pp._raw_spin_lock_irqsave
      0.92 ±  9%      +0.5        1.45 ± 10%  perf-profile.children.cycles-pp.__mem_cgroup_flush_stats
      0.92 ± 10%      +0.5        1.45 ± 10%  perf-profile.children.cycles-pp.cgroup_rstat_flush_irqsafe
      0.92 ±  9%      +0.5        1.44 ± 10%  perf-profile.children.cycles-pp.cgroup_rstat_flush_locked
      0.94 ± 10%      +0.5        1.48 ± 10%  perf-profile.children.cycles-pp.mem_cgroup_wb_stats
      0.95 ± 10%      +0.5        1.50 ± 10%  perf-profile.children.cycles-pp.balance_dirty_pages
      0.98 ± 10%      +0.6        1.54 ± 10%  perf-profile.children.cycles-pp.balance_dirty_pages_ratelimited
      4.71 ±  7%      +0.8        5.52 ±  4%  perf-profile.children.cycles-pp.btrfs_create_common
      6.00 ±  7%      +0.8        6.84 ±  4%  perf-profile.children.cycles-pp.__x64_sys_openat
      5.99 ±  7%      +0.8        6.83 ±  4%  perf-profile.children.cycles-pp.do_sys_openat2
      5.93 ±  7%      +0.8        6.76 ±  4%  perf-profile.children.cycles-pp.do_filp_open
      5.92 ±  7%      +0.8        6.76 ±  4%  perf-profile.children.cycles-pp.path_openat
      6.01 ±  7%      +0.8        6.85 ±  4%  perf-profile.children.cycles-pp.open64
      5.58 ±  6%      +0.9        6.44 ±  4%  perf-profile.children.cycles-pp.lookup_open
      5.63 ±  7%      +0.9        6.50 ±  4%  perf-profile.children.cycles-pp.open_last_lookups
      0.57 ± 70%      +0.9        1.46 ± 25%  perf-profile.children.cycles-pp.timekeeping_max_deferment
      1.26 ± 10%      +1.0        2.27 ± 10%  perf-profile.children.cycles-pp.copy_page
      1.26 ± 10%      +1.0        2.27 ± 10%  perf-profile.children.cycles-pp.copy_extent_buffer_full
      1.56 ± 28%      +1.1        2.65 ± 14%  perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
      0.84 ±  8%      +1.1        1.98 ±  4%  perf-profile.children.cycles-pp.start_log_trans
      1.20 ± 38%      +1.2        2.36 ± 16%  perf-profile.children.cycles-pp.tick_nohz_next_event
      3.13 ±  9%      +1.9        5.00 ±  5%  perf-profile.children.cycles-pp.__btrfs_cow_block
      3.15 ±  9%      +1.9        5.03 ±  5%  perf-profile.children.cycles-pp.btrfs_cow_block
      5.98 ±  6%      +2.0        8.02 ±  6%  perf-profile.children.cycles-pp.btrfs_insert_empty_items
      3.07 ±  6%      +2.2        5.29 ±  6%  perf-profile.children.cycles-pp.copy_items
      4.44 ±  7%      +2.3        6.76 ±  5%  perf-profile.children.cycles-pp.copy_inode_items_to_log
      5.24 ±  7%      +2.8        8.02 ±  4%  perf-profile.children.cycles-pp.btrfs_log_inode
     22.41 ±  8%      +3.4       25.83 ±  3%  perf-profile.children.cycles-pp.fsync
      9.25 ±  8%      +3.4       12.68 ±  4%  perf-profile.children.cycles-pp.btrfs_search_slot
     22.29 ±  8%      +3.4       25.72 ±  3%  perf-profile.children.cycles-pp.__x64_sys_fsync
     22.28 ±  8%      +3.4       25.71 ±  3%  perf-profile.children.cycles-pp.btrfs_sync_file
      6.19 ±  7%      +4.0       10.18 ±  3%  perf-profile.children.cycles-pp.btrfs_log_inode_parent
      6.20 ±  7%      +4.0       10.20 ±  3%  perf-profile.children.cycles-pp.btrfs_log_dentry_safe
     31.26 ±  7%      +4.4       35.71 ±  3%  perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
     31.24 ±  7%      +4.4       35.69 ±  3%  perf-profile.children.cycles-pp.do_syscall_64
      1.76 ±  5%      -0.3        1.46 ±  9%  perf-profile.self.cycles-pp.cpuidle_enter_state
      0.14 ± 34%      -0.1        0.08 ±  9%  perf-profile.self.cycles-pp.nr_iowait_cpu
      0.11 ± 43%      -0.1        0.04 ± 75%  perf-profile.self.cycles-pp.scheduler_tick
      0.22 ± 17%      -0.1        0.16 ±  9%  perf-profile.self.cycles-pp.__hrtimer_next_event_base
      0.14 ± 18%      -0.0        0.10 ± 14%  perf-profile.self.cycles-pp.perf_mux_hrtimer_handler
      0.08 ± 23%      -0.0        0.03 ±101%  perf-profile.self.cycles-pp.error_entry
      0.08 ± 18%      -0.0        0.04 ± 71%  perf-profile.self.cycles-pp.clockevents_program_event
      0.16 ±  9%      -0.0        0.12 ± 13%  perf-profile.self.cycles-pp.do_idle
      0.06 ± 23%      +0.0        0.09 ± 12%  perf-profile.self.cycles-pp.up_write
      0.07 ± 10%      +0.0        0.10 ± 21%  perf-profile.self.cycles-pp.end_bio_extent_buffer_writepage
      0.06 ± 17%      +0.0        0.10 ±  6%  perf-profile.self.cycles-pp.set_extent_buffer_dirty
      0.02 ±141%      +0.0        0.06 ± 18%  perf-profile.self.cycles-pp.__mark_inode_dirty
      0.04 ± 73%      +0.0        0.09 ± 17%  perf-profile.self.cycles-pp.start_transaction
      0.09 ± 20%      +0.0        0.13 ± 13%  perf-profile.self.cycles-pp.percpu_counter_add_batch
      0.09 ± 19%      +0.0        0.14 ±  9%  perf-profile.self.cycles-pp.up_read
      0.02 ±142%      +0.0        0.06 ± 11%  perf-profile.self.cycles-pp.btrfs_clean_tree_block
      0.16 ± 15%      +0.0        0.21 ±  9%  perf-profile.self.cycles-pp.set_extent_bit
      0.01 ±223%      +0.0        0.06 ± 13%  perf-profile.self.cycles-pp.available_idle_cpu
      0.02 ±141%      +0.1        0.07 ± 14%  perf-profile.self.cycles-pp.xas_set_mark
      0.06 ± 17%      +0.1        0.12 ±  9%  perf-profile.self.cycles-pp.mark_page_accessed
      0.08 ± 10%      +0.1        0.13 ± 14%  perf-profile.self.cycles-pp.__mod_memcg_lruvec_state
      0.09 ± 24%      +0.1        0.14 ± 13%  perf-profile.self.cycles-pp.free_extent_buffer
      0.02 ± 99%      +0.1        0.08 ± 13%  perf-profile.self.cycles-pp.rwsem_mark_wake
      0.18 ±  5%      +0.1        0.25 ± 19%  perf-profile.self.cycles-pp.__btrfs_cow_block
      0.00            +0.1        0.08 ±  9%  perf-profile.self.cycles-pp.verify_parent_transid
      0.00            +0.1        0.08 ± 19%  perf-profile.self.cycles-pp.llist_add_batch
      0.01 ±223%      +0.1        0.09 ± 14%  perf-profile.self.cycles-pp.btrfs_sync_log
      0.01 ±223%      +0.1        0.09 ± 14%  perf-profile.self.cycles-pp.need_preemptive_reclaim
      0.25 ± 15%      +0.1        0.33 ± 21%  perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
      0.16 ±  9%      +0.1        0.26 ± 10%  perf-profile.self.cycles-pp.btrfs_root_node
      0.10 ± 19%      +0.1        0.19 ± 10%  perf-profile.self.cycles-pp.__filemap_get_folio
      0.02 ±145%      +0.1        0.12 ±  9%  perf-profile.self.cycles-pp.btrfs_buffer_uptodate
      0.21 ± 14%      +0.1        0.32 ± 12%  perf-profile.self.cycles-pp.down_read
      0.17 ± 18%      +0.1        0.28 ± 10%  perf-profile.self.cycles-pp.mutex_lock
      0.12 ± 11%      +0.1        0.24 ± 11%  perf-profile.self.cycles-pp.inode_tree_add
      0.65 ±  9%      +0.1        0.77 ±  4%  perf-profile.self.cycles-pp.generic_bin_search
      0.30 ± 15%      +0.1        0.44 ±  4%  perf-profile.self.cycles-pp.btrfs_search_slot
      0.40 ±  9%      +0.2        0.56 ± 10%  perf-profile.self.cycles-pp.mem_cgroup_css_rstat_flush
      0.34 ± 15%      +0.2        0.50 ±  7%  perf-profile.self.cycles-pp.find_extent_buffer_nolock
      0.17 ± 33%      +0.2        0.34 ± 24%  perf-profile.self.cycles-pp.tsc_verify_tsc_adjust
      0.45 ± 14%      +0.3        0.72 ±  8%  perf-profile.self.cycles-pp.rwsem_spin_on_owner
      0.30 ± 34%      +0.4        0.65 ± 19%  perf-profile.self.cycles-pp.tick_nohz_next_event
      1.01 ±  4%      +0.5        1.47 ±  9%  perf-profile.self.cycles-pp._raw_spin_lock_irqsave
      0.56 ± 74%      +0.9        1.46 ± 25%  perf-profile.self.cycles-pp.timekeeping_max_deferment
      1.24 ± 10%      +1.0        2.24 ± 10%  perf-profile.self.cycles-pp.copy_page





Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


-- 
0-DAY CI Kernel Test Service
https://01.org/lkp



View attachment "config-5.19.0-rc2-00003-gcb29a5c19d2d" of type "text/plain" (163951 bytes)

View attachment "job-script" of type "text/plain" (7702 bytes)

View attachment "job.yaml" of type "text/plain" (5036 bytes)

View attachment "reproduce" of type "text/plain" (298 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ