lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-ID: <20201110060012.GA3197@xsang-OptiPlex-9020>
Date:   Tue, 10 Nov 2020 14:00:12 +0800
From:   kernel test robot <oliver.sang@...el.com>
To:     Josef Bacik <josef@...icpanda.com>
Cc:     David Sterba <dsterba@...e.com>,
        LKML <linux-kernel@...r.kernel.org>,
        Linux Memory Management List <linux-mm@...ck.org>,
        lkp@...ts.01.org, lkp@...el.com, ying.huang@...el.com,
        feng.tang@...el.com, zhengjun.xing@...el.com,
        linux-btrfs@...r.kernel.org
Subject: [btrfs]  fac2f60d5f:  fsmark.files_per_sec 62.6% improvement

Greeting,

FYI, we noticed a 62.6% improvement of fsmark.files_per_sec due to commit:


commit: fac2f60d5fe83fd45ee08a85c2eb7a09659edbe3 ("btrfs: switch extent buffer tree lock to rw_semaphore")
https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master


in testcase: fsmark
on test machine: 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 512G memory
with following parameters:

	iterations: 1x
	nr_threads: 32t
	disk: 1SSD
	fs: btrfs
	filesize: 9B
	test_size: 400M
	sync_method: fsyncBeforeClose
	nr_directories: 16d
	nr_files_per_directory: 256fpd
	cpufreq_governor: performance
	ucode: 0x5002f01

test-description: The fsmark is a file system benchmark to test synchronous write workloads, for example, mail servers workload.
test-url: https://sourceforge.net/projects/fsmark/

In addition to that, the commit also has significant impact on the following tests:

+------------------+----------------------------------------------------------------------+
| testcase: change | fsmark: fsmark.files_per_sec 42.6% improvement                       |
| test machine     | 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 512G memory |
| test parameters  | cpufreq_governor=performance                                         |
|                  | disk=1SSD                                                            |
|                  | filesize=8K                                                          |
|                  | fs=btrfs                                                             |
|                  | iterations=1x                                                        |
|                  | nr_directories=16d                                                   |
|                  | nr_files_per_directory=256fpd                                        |
|                  | nr_threads=32t                                                       |
|                  | sync_method=fsyncBeforeClose                                         |
|                  | test_size=400M                                                       |
|                  | ucode=0x5002f01                                                      |
+------------------+----------------------------------------------------------------------+




Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp install job.yaml  # job file is attached in this email
        bin/lkp run     job.yaml

=========================================================================================
compiler/cpufreq_governor/disk/filesize/fs/iterations/kconfig/nr_directories/nr_files_per_directory/nr_threads/rootfs/sync_method/tbox_group/test_size/testcase/ucode:
  gcc-9/performance/1SSD/9B/btrfs/1x/x86_64-rhel-8.3/16d/256fpd/32t/debian-10.4-x86_64-20200603.cgz/fsyncBeforeClose/lkp-csl-2sp7/400M/fsmark/0x5002f01

commit: 
  b010373b40 ("btrfs: fix potential overflow in cluster_pages_for_defrag on 32bit arch")
  fac2f60d5f ("btrfs: switch extent buffer tree lock to rw_semaphore")

b010373b4014116a fac2f60d5fe83fd45ee08a85c2e 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
   1111549 ±  3%      -8.2%    1020575 ±  5%  fsmark.app_overhead
      8149 ±  2%     +62.6%      13254 ±  5%  fsmark.files_per_sec
     13.56 ±  2%     -35.3%       8.77 ±  4%  fsmark.time.elapsed_time
     13.56 ±  2%     -35.3%       8.77 ±  4%  fsmark.time.elapsed_time.max
   2365160 ±  3%      -8.4%    2165632 ±  2%  fsmark.time.file_system_outputs
      1073           -37.0%     676.00        fsmark.time.percent_of_cpu_this_job_got
    144.77           -59.6%      58.51 ±  3%  fsmark.time.system_time
   2421162 ±  2%     -66.0%     824094 ±  7%  fsmark.time.voluntary_context_switches
     88.93            +3.7%      92.23        iostat.cpu.idle
      9.91 ±  2%     -34.0%       6.54 ±  4%  iostat.cpu.system
    253958           -23.3%     194817 ±  3%  softirqs.RCU
    323262           -18.2%     264378 ±  3%  softirqs.SCHED
    558709 ± 92%     -93.8%      34831 ±  9%  sched_debug.cfs_rq:/.load.max
   1783705 ±  6%     -32.4%    1205297 ± 21%  sched_debug.cpu.avg_idle.max
     78002 ± 27%     -45.0%      42913 ± 12%  sched_debug.cpu.max_idle_balance_cost.stddev
      0.06 ± 14%      +0.0        0.09 ±  5%  mpstat.cpu.all.iowait%
      0.74            +0.3        0.99 ± 15%  mpstat.cpu.all.irq%
      0.08            +0.0        0.11 ±  7%  mpstat.cpu.all.soft%
     10.42            -4.2        6.25 ±  5%  mpstat.cpu.all.sys%
     88.50            +3.7%      91.75        vmstat.cpu.id
     46998 ±  3%     +22.7%      57683 ±  9%  vmstat.io.bo
    295884 ±  3%     -51.2%     144316 ±  4%  vmstat.system.cs
    184790 ±  2%      -3.7%     177862        vmstat.system.in
 1.214e+08           -91.4%   10386665        cpuidle.C1.time
   1215140 ±  3%     -88.0%     145379 ±  2%  cpuidle.C1.usage
 1.117e+09 ±  3%     -32.5%  7.544e+08 ± 36%  cpuidle.C1E.time
   3631170 ±  3%     -35.5%    2343511 ± 17%  cpuidle.C1E.usage
     26134 ±  9%     +67.7%      43814 ± 13%  cpuidle.POLL.usage
    198981 ± 14%     -30.3%     138691 ± 22%  numa-meminfo.node0.Active
    197944 ± 15%     -30.6%     137337 ± 22%  numa-meminfo.node0.Active(file)
      7830 ± 36%     -73.6%       2070 ± 17%  numa-meminfo.node0.Shmem
     15277 ± 14%     +25.3%      19148        numa-meminfo.node1.Mapped
      3697 ± 77%    +152.0%       9316 ±  3%  numa-meminfo.node1.Shmem
    236590 ±  6%     -29.9%     165861 ± 12%  meminfo.Active
    234474 ±  6%     -30.1%     163902 ± 12%  meminfo.Active(file)
    278019 ±  2%      -8.4%     254569 ±  5%  meminfo.Inactive(file)
    145261 ±  2%      -8.8%     132486 ±  2%  meminfo.KReclaimable
      9433 ±  2%     +21.8%      11493 ±  6%  meminfo.PageTables
    145261 ±  2%      -8.8%     132486 ±  2%  meminfo.SReclaimable
    243266           +35.5%     329601 ±  7%  meminfo.max_used_kB
     48521 ± 11%     -30.3%      33818 ± 23%  numa-vmstat.node0.nr_active_file
      1957 ± 36%     -73.6%     517.25 ± 17%  numa-vmstat.node0.nr_shmem
     48521 ± 11%     -30.3%      33818 ± 23%  numa-vmstat.node0.nr_zone_active_file
     56472 ± 73%    +144.6%     138134 ±  6%  numa-vmstat.node0.numa_other
      3862 ± 13%     +25.0%       4828        numa-vmstat.node1.nr_mapped
    924.00 ± 77%    +152.1%       2329 ±  3%  numa-vmstat.node1.nr_shmem
    132552 ± 31%     -62.4%      49788 ± 19%  numa-vmstat.node1.numa_other
     99918 ±  3%     -13.5%      86390 ±  4%  slabinfo.Acpi-State.active_objs
      1972 ±  3%     -13.3%       1709 ±  4%  slabinfo.Acpi-State.active_slabs
    100598 ±  3%     -13.3%      87206 ±  4%  slabinfo.Acpi-State.num_objs
      1972 ±  3%     -13.3%       1709 ±  4%  slabinfo.Acpi-State.num_slabs
     49053 ±  3%     -13.2%      42571 ±  4%  slabinfo.btrfs_delayed_node.active_objs
    944.25 ±  4%     -13.2%     820.00 ±  4%  slabinfo.btrfs_delayed_node.active_slabs
     49133 ±  4%     -13.2%      42646 ±  4%  slabinfo.btrfs_delayed_node.num_objs
    944.25 ±  4%     -13.2%     820.00 ±  4%  slabinfo.btrfs_delayed_node.num_slabs
     48487 ±  4%     -13.6%      41914 ±  4%  slabinfo.btrfs_inode.active_objs
      1732 ±  4%     -13.5%       1498 ±  4%  slabinfo.btrfs_inode.active_slabs
     48511 ±  4%     -13.5%      41960 ±  4%  slabinfo.btrfs_inode.num_objs
      1732 ±  4%     -13.5%       1498 ±  4%  slabinfo.btrfs_inode.num_slabs
      6772 ±  4%     -97.3%     186.00        slabinfo.numa_policy.active_objs
    108.75 ±  3%     -97.2%       3.00        slabinfo.numa_policy.active_slabs
      6772 ±  4%     -97.3%     186.00        slabinfo.numa_policy.num_objs
    108.75 ±  3%     -97.2%       3.00        slabinfo.numa_policy.num_slabs
      5.41 ± 71%      -3.3        2.08 ±173%  perf-profile.calltrace.cycles-pp.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe
      5.41 ± 71%      -3.3        2.08 ±173%  perf-profile.calltrace.cycles-pp.mmap_region.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64
      4.54 ±100%      -2.5        2.08 ±173%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__mmap
      4.54 ±100%      -2.5        2.08 ±173%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__mmap
      4.54 ±100%      -2.5        2.08 ±173%  perf-profile.calltrace.cycles-pp.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe.__mmap
      4.54 ±100%      -2.5        2.08 ±173%  perf-profile.calltrace.cycles-pp.__mmap
      4.54 ±100%      -2.5        2.08 ±173%  perf-profile.calltrace.cycles-pp.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe.__mmap
      4.06 ±101%      -2.0        2.08 ±173%  perf-profile.calltrace.cycles-pp.mmput.do_exit.do_group_exit.get_signal.arch_do_signal
      4.06 ±101%      -2.0        2.08 ±173%  perf-profile.calltrace.cycles-pp.exit_mmap.mmput.do_exit.do_group_exit.get_signal
      4.54 ±100%      -4.5        0.00        perf-profile.children.cycles-pp.terminate_walk
      4.54 ±100%      -4.3        0.24 ±173%  perf-profile.children.cycles-pp.dput
      5.41 ± 71%      -3.3        2.08 ±173%  perf-profile.children.cycles-pp.do_mmap
      5.41 ± 71%      -3.3        2.08 ±173%  perf-profile.children.cycles-pp.ksys_mmap_pgoff
      5.41 ± 71%      -3.3        2.08 ±173%  perf-profile.children.cycles-pp.mmap_region
      5.41 ± 71%      -3.3        2.08 ±173%  perf-profile.children.cycles-pp.vm_mmap_pgoff
      4.54 ±100%      -2.5        2.08 ±173%  perf-profile.children.cycles-pp.__mmap
      4.06 ±101%      -2.0        2.08 ±173%  perf-profile.children.cycles-pp.kmem_cache_alloc
    528.50            -7.4%     489.25 ±  2%  proc-vmstat.nr_active_anon
     58090 ±  4%     -30.4%      40420 ± 12%  proc-vmstat.nr_active_file
     69130            +4.6%      72337        proc-vmstat.nr_anon_pages
    295117 ±  2%      -8.3%     270555 ±  2%  proc-vmstat.nr_dirtied
    387199            -5.9%     364434 ±  2%  proc-vmstat.nr_file_pages
     71383            +4.5%      74599        proc-vmstat.nr_inactive_anon
     69177 ±  3%      -8.3%      63402 ±  6%  proc-vmstat.nr_inactive_file
     19128            +1.9%      19492        proc-vmstat.nr_kernel_stack
      2358 ±  2%     +21.7%       2868 ±  6%  proc-vmstat.nr_page_table_pages
      2881            -1.2%       2846        proc-vmstat.nr_shmem
     36346 ±  2%      -9.3%      32980 ±  2%  proc-vmstat.nr_slab_reclaimable
    295151 ±  2%      -8.4%     270491 ±  2%  proc-vmstat.nr_written
    528.50            -7.4%     489.25 ±  2%  proc-vmstat.nr_zone_active_anon
     58090 ±  4%     -30.4%      40420 ± 12%  proc-vmstat.nr_zone_active_file
     71383            +4.5%      74599        proc-vmstat.nr_zone_inactive_anon
     69177 ±  3%      -8.3%      63402 ±  6%  proc-vmstat.nr_zone_inactive_file
    502830            -6.9%     468198        proc-vmstat.numa_hit
    471656            -7.3%     437027        proc-vmstat.numa_local
     22135            -9.1%      20122 ±  3%  proc-vmstat.pgactivate
    529483            -7.8%     488229        proc-vmstat.pgalloc_normal
    123890           -11.0%     110251 ±  3%  proc-vmstat.pgfault
    259358 ± 34%     +66.9%     432965 ± 18%  proc-vmstat.pgfree
    786773 ±  4%     -12.5%     688368 ±  4%  proc-vmstat.pgpgout
      5628 ±  2%     -14.1%       4834 ±  3%  proc-vmstat.pgreuse
     10.41 ±  4%     -23.1%       8.00 ±  9%  perf-stat.i.MPKI
 1.297e+08 ±  4%     -18.5%  1.057e+08 ± 11%  perf-stat.i.cache-references
    359955           -48.0%     187045 ±  4%  perf-stat.i.context-switches
      2.96           -36.1%       1.89 ±  2%  perf-stat.i.cpi
 3.695e+10           -32.0%  2.511e+10 ±  5%  perf-stat.i.cpu-cycles
      0.02 ±  2%      +0.0        0.02 ± 34%  perf-stat.i.dTLB-load-miss-rate%
    466201 ±  2%     +59.6%     744250 ± 36%  perf-stat.i.dTLB-load-misses
      0.01 ±  2%      +0.0        0.01 ± 26%  perf-stat.i.dTLB-store-miss-rate%
     66981 ±  2%     +73.8%     116408 ± 28%  perf-stat.i.dTLB-store-misses
   4122745 ±  2%     -10.4%    3695563 ±  3%  perf-stat.i.iTLB-load-misses
   7202794            -6.4%    6740171 ±  2%  perf-stat.i.iTLB-loads
      3095 ±  2%     +17.0%       3623 ±  5%  perf-stat.i.instructions-per-iTLB-miss
      0.38           +49.7%       0.57 ±  2%  perf-stat.i.ipc
      0.38           -32.0%       0.26 ±  5%  perf-stat.i.metric.GHz
      7960 ±  3%     +27.9%      10184 ±  2%  perf-stat.i.minor-faults
    307681 ± 13%     +63.3%     502376 ±  7%  perf-stat.i.node-stores
      7961 ±  3%     +27.9%      10185 ±  2%  perf-stat.i.page-faults
     10.30 ±  4%     -23.0%       7.93 ±  8%  perf-stat.overall.MPKI
      2.94           -35.7%       1.89        perf-stat.overall.cpi
      0.02 ±  2%      +0.0        0.02 ± 32%  perf-stat.overall.dTLB-load-miss-rate%
      0.01 ±  2%      +0.0        0.01 ± 25%  perf-stat.overall.dTLB-store-miss-rate%
     36.40            -1.0       35.41        perf-stat.overall.iTLB-load-miss-rate%
      3056 ±  2%     +17.8%       3600 ±  5%  perf-stat.overall.instructions-per-iTLB-miss
      0.34           +55.4%       0.53        perf-stat.overall.ipc
 1.206e+08 ±  4%     -21.2%   95004833 ± 11%  perf-stat.ps.cache-references
    334704           -49.7%     168237 ±  4%  perf-stat.ps.context-switches
     89285            -3.3%      86363        perf-stat.ps.cpu-clock
 3.438e+10           -34.3%  2.258e+10 ±  4%  perf-stat.ps.cpu-cycles
    433642 ±  2%     +54.1%     668298 ± 35%  perf-stat.ps.dTLB-load-misses
     62411 ±  2%     +67.6%     104604 ± 27%  perf-stat.ps.dTLB-store-misses
   3834185 ±  2%     -13.3%    3325031 ±  3%  perf-stat.ps.iTLB-load-misses
   6698496            -9.5%    6063638 ±  2%  perf-stat.ps.iTLB-loads
      7391 ±  3%     +23.9%       9159 ±  2%  perf-stat.ps.minor-faults
    286228 ± 13%     +57.8%     451688 ±  6%  perf-stat.ps.node-stores
      7392 ±  3%     +23.9%       9160 ±  2%  perf-stat.ps.page-faults
     89285            -3.3%      86363        perf-stat.ps.task-clock
  1.69e+11 ±  3%     -28.8%  1.203e+11 ±  3%  perf-stat.total.instructions
    152404 ±  5%     -49.9%      76304        interrupts.CAL:Function_call_interrupts
      2754 ± 14%     -67.8%     888.00 ± 51%  interrupts.CPU0.CAL:Function_call_interrupts
     30120           -30.3%      20996 ±  5%  interrupts.CPU0.LOC:Local_timer_interrupts
    250.50 ± 21%     -75.2%      62.00 ± 15%  interrupts.CPU0.RES:Rescheduling_interrupts
      3387 ± 27%     -62.7%       1263 ± 31%  interrupts.CPU1.CAL:Function_call_interrupts
     30104           -31.1%      20748 ±  8%  interrupts.CPU1.LOC:Local_timer_interrupts
    364.50 ± 28%     -81.7%      66.75 ± 34%  interrupts.CPU1.RES:Rescheduling_interrupts
      1850 ± 12%     -47.5%     971.75 ± 15%  interrupts.CPU10.CAL:Function_call_interrupts
     30081           -30.4%      20937 ±  5%  interrupts.CPU10.LOC:Local_timer_interrupts
    216.25 ± 18%     -61.7%      82.75 ± 31%  interrupts.CPU10.RES:Rescheduling_interrupts
      2506 ± 10%     -64.0%     903.00 ±  6%  interrupts.CPU11.CAL:Function_call_interrupts
     30066           -29.8%      21107 ±  6%  interrupts.CPU11.LOC:Local_timer_interrupts
    359.00 ± 20%     -83.4%      59.75 ± 15%  interrupts.CPU11.RES:Rescheduling_interrupts
      2464 ± 25%     -61.0%     961.00 ±  4%  interrupts.CPU12.CAL:Function_call_interrupts
     30063           -30.0%      21050 ±  6%  interrupts.CPU12.LOC:Local_timer_interrupts
    326.00 ± 26%     -77.8%      72.50 ± 12%  interrupts.CPU12.RES:Rescheduling_interrupts
      2060 ± 26%     -54.3%     942.50 ± 13%  interrupts.CPU13.CAL:Function_call_interrupts
     29932 ±  2%     -29.5%      21105 ±  6%  interrupts.CPU13.LOC:Local_timer_interrupts
    258.75 ± 33%     -68.6%      81.25 ± 22%  interrupts.CPU13.RES:Rescheduling_interrupts
      2226 ± 34%     -49.9%       1114 ± 10%  interrupts.CPU14.CAL:Function_call_interrupts
     30075           -29.8%      21111 ±  6%  interrupts.CPU14.LOC:Local_timer_interrupts
    277.50 ± 43%     -67.8%      89.25 ± 19%  interrupts.CPU14.RES:Rescheduling_interrupts
      2581 ± 17%     -68.7%     808.50 ± 30%  interrupts.CPU15.CAL:Function_call_interrupts
     29982 ±  2%     -30.1%      20971 ±  6%  interrupts.CPU15.LOC:Local_timer_interrupts
    320.75 ± 22%     -74.1%      83.00 ± 22%  interrupts.CPU15.RES:Rescheduling_interrupts
      3101 ± 25%     -70.6%     912.00 ± 20%  interrupts.CPU16.CAL:Function_call_interrupts
     30034           -30.9%      20766 ±  7%  interrupts.CPU16.LOC:Local_timer_interrupts
    278.00 ± 27%     -72.6%      76.25 ± 16%  interrupts.CPU16.RES:Rescheduling_interrupts
     30061           -30.2%      20991 ±  6%  interrupts.CPU17.LOC:Local_timer_interrupts
     30048           -29.7%      21122 ±  6%  interrupts.CPU18.LOC:Local_timer_interrupts
     30059           -29.9%      21058 ±  6%  interrupts.CPU19.LOC:Local_timer_interrupts
      2960 ± 44%     -68.6%     928.25 ± 12%  interrupts.CPU2.CAL:Function_call_interrupts
     29980 ±  2%     -29.7%      21065 ±  6%  interrupts.CPU2.LOC:Local_timer_interrupts
    317.75 ± 35%     -77.3%      72.00 ± 26%  interrupts.CPU2.RES:Rescheduling_interrupts
      2478 ± 33%     -64.0%     893.00 ± 14%  interrupts.CPU20.CAL:Function_call_interrupts
     29995           -29.7%      21082 ±  6%  interrupts.CPU20.LOC:Local_timer_interrupts
    317.00 ± 44%     -82.9%      54.25 ± 47%  interrupts.CPU20.RES:Rescheduling_interrupts
      2356 ± 29%     -53.6%       1093 ± 28%  interrupts.CPU21.CAL:Function_call_interrupts
     30051           -29.9%      21054 ±  6%  interrupts.CPU21.LOC:Local_timer_interrupts
    278.00 ± 32%     -71.0%      80.50 ± 31%  interrupts.CPU21.RES:Rescheduling_interrupts
      2209 ± 43%     -60.3%     876.25 ± 17%  interrupts.CPU22.CAL:Function_call_interrupts
     30063           -29.8%      21112 ±  6%  interrupts.CPU22.LOC:Local_timer_interrupts
    291.50 ± 57%     -78.6%      62.50 ± 18%  interrupts.CPU22.RES:Rescheduling_interrupts
      2241 ± 22%     -53.8%       1035 ± 20%  interrupts.CPU23.CAL:Function_call_interrupts
     30081           -30.0%      21049 ±  6%  interrupts.CPU23.LOC:Local_timer_interrupts
    296.50 ± 20%     -72.3%      82.00 ± 27%  interrupts.CPU23.RES:Rescheduling_interrupts
     30057 ±  2%     -30.4%      20905 ±  7%  interrupts.CPU24.LOC:Local_timer_interrupts
      1056 ± 27%     -48.5%     544.50 ± 15%  interrupts.CPU25.CAL:Function_call_interrupts
     29999 ±  2%     -29.7%      21102 ±  6%  interrupts.CPU25.LOC:Local_timer_interrupts
     29686 ±  2%     -29.0%      21082 ±  6%  interrupts.CPU26.LOC:Local_timer_interrupts
     30033 ±  2%     -30.0%      21025 ±  6%  interrupts.CPU27.LOC:Local_timer_interrupts
     30048 ±  2%     -29.8%      21079 ±  6%  interrupts.CPU28.LOC:Local_timer_interrupts
     29785 ±  2%     -29.5%      20984 ±  6%  interrupts.CPU29.LOC:Local_timer_interrupts
      2767 ± 17%     -63.9%     998.50 ± 20%  interrupts.CPU3.CAL:Function_call_interrupts
     30050           -29.8%      21107 ±  6%  interrupts.CPU3.LOC:Local_timer_interrupts
    361.00 ± 22%     -80.5%      70.25 ± 40%  interrupts.CPU3.RES:Rescheduling_interrupts
     29993           -29.8%      21040 ±  6%  interrupts.CPU30.LOC:Local_timer_interrupts
     30207           -30.3%      21052 ±  6%  interrupts.CPU31.LOC:Local_timer_interrupts
     30029           -29.9%      21057 ±  6%  interrupts.CPU32.LOC:Local_timer_interrupts
     29887           -29.5%      21059 ±  6%  interrupts.CPU33.LOC:Local_timer_interrupts
     29981 ±  2%     -29.7%      21078 ±  6%  interrupts.CPU34.LOC:Local_timer_interrupts
     29692           -29.1%      21066 ±  6%  interrupts.CPU35.LOC:Local_timer_interrupts
     30009           -29.8%      21053 ±  6%  interrupts.CPU36.LOC:Local_timer_interrupts
     29988           -29.9%      21019 ±  6%  interrupts.CPU37.LOC:Local_timer_interrupts
     30027           -29.9%      21048 ±  6%  interrupts.CPU38.LOC:Local_timer_interrupts
     30008 ±  2%     -29.8%      21052 ±  6%  interrupts.CPU39.LOC:Local_timer_interrupts
      2795 ± 15%     -58.8%       1151 ± 29%  interrupts.CPU4.CAL:Function_call_interrupts
     30055           -30.0%      21045 ±  6%  interrupts.CPU4.LOC:Local_timer_interrupts
    337.75 ± 13%     -80.2%      67.00 ± 42%  interrupts.CPU4.RES:Rescheduling_interrupts
     30019           -29.9%      21035 ±  6%  interrupts.CPU40.LOC:Local_timer_interrupts
     30042           -29.9%      21055 ±  6%  interrupts.CPU41.LOC:Local_timer_interrupts
     29988 ±  2%     -29.8%      21056 ±  6%  interrupts.CPU42.LOC:Local_timer_interrupts
     29965           -29.8%      21042 ±  6%  interrupts.CPU43.LOC:Local_timer_interrupts
     30005 ±  2%     -29.9%      21026 ±  6%  interrupts.CPU44.LOC:Local_timer_interrupts
     30056           -29.9%      21056 ±  6%  interrupts.CPU45.LOC:Local_timer_interrupts
     30001           -29.9%      21043 ±  6%  interrupts.CPU46.LOC:Local_timer_interrupts
     29986 ±  2%     -29.7%      21078 ±  6%  interrupts.CPU47.LOC:Local_timer_interrupts
      1844 ± 12%     -47.7%     964.25 ± 26%  interrupts.CPU48.CAL:Function_call_interrupts
     30050           -30.0%      21027 ±  6%  interrupts.CPU48.LOC:Local_timer_interrupts
    229.75 ± 19%     -69.9%      69.25 ± 41%  interrupts.CPU48.RES:Rescheduling_interrupts
      2444 ± 28%     -65.5%     843.00 ± 16%  interrupts.CPU49.CAL:Function_call_interrupts
     30077           -29.9%      21097 ±  6%  interrupts.CPU49.LOC:Local_timer_interrupts
    297.25 ± 29%     -79.6%      60.75 ± 42%  interrupts.CPU49.RES:Rescheduling_interrupts
      2547 ± 31%     -55.7%       1127 ± 27%  interrupts.CPU5.CAL:Function_call_interrupts
     30068           -29.7%      21128 ±  6%  interrupts.CPU5.LOC:Local_timer_interrupts
    362.25 ± 38%     -78.7%      77.25 ± 36%  interrupts.CPU5.RES:Rescheduling_interrupts
      2085 ± 20%     -56.5%     907.00 ± 19%  interrupts.CPU50.CAL:Function_call_interrupts
     30069           -29.9%      21071 ±  6%  interrupts.CPU50.LOC:Local_timer_interrupts
    270.75 ± 29%     -75.5%      66.25 ± 42%  interrupts.CPU50.RES:Rescheduling_interrupts
      2689 ± 26%     -64.3%     960.00 ± 24%  interrupts.CPU51.CAL:Function_call_interrupts
     30084           -29.2%      21299 ±  6%  interrupts.CPU51.LOC:Local_timer_interrupts
    349.00 ± 30%     -81.5%      64.50 ± 35%  interrupts.CPU51.RES:Rescheduling_interrupts
      2423 ± 25%     -61.6%     931.00 ±  4%  interrupts.CPU52.CAL:Function_call_interrupts
     30071           -29.8%      21120 ±  6%  interrupts.CPU52.LOC:Local_timer_interrupts
    291.75 ± 32%     -73.9%      76.25 ± 14%  interrupts.CPU52.RES:Rescheduling_interrupts
      2067 ± 36%     -61.7%     792.50 ± 16%  interrupts.CPU53.CAL:Function_call_interrupts
     30071           -30.1%      21024 ±  6%  interrupts.CPU53.LOC:Local_timer_interrupts
    258.00 ± 47%     -74.4%      66.00 ± 30%  interrupts.CPU53.RES:Rescheduling_interrupts
      2839 ±  9%     -67.0%     938.50 ± 12%  interrupts.CPU54.CAL:Function_call_interrupts
     30099           -29.9%      21109 ±  6%  interrupts.CPU54.LOC:Local_timer_interrupts
    388.25 ± 14%     -80.4%      76.25 ± 19%  interrupts.CPU54.RES:Rescheduling_interrupts
      2524 ± 26%     -65.8%     862.50 ± 11%  interrupts.CPU55.CAL:Function_call_interrupts
     30059           -29.8%      21107 ±  6%  interrupts.CPU55.LOC:Local_timer_interrupts
    331.50 ± 32%     -84.4%      51.75 ± 13%  interrupts.CPU55.RES:Rescheduling_interrupts
      2397 ± 26%     -70.4%     709.00 ± 18%  interrupts.CPU56.CAL:Function_call_interrupts
     30051           -29.7%      21115 ±  6%  interrupts.CPU56.LOC:Local_timer_interrupts
    304.50 ± 27%     -81.5%      56.25 ± 32%  interrupts.CPU56.RES:Rescheduling_interrupts
      2955 ± 17%     -63.2%       1088 ± 29%  interrupts.CPU57.CAL:Function_call_interrupts
     30026           -29.7%      21102 ±  6%  interrupts.CPU57.LOC:Local_timer_interrupts
    402.50 ± 22%     -79.5%      82.50 ± 54%  interrupts.CPU57.RES:Rescheduling_interrupts
      2827 ± 17%     -67.9%     907.00 ± 30%  interrupts.CPU58.CAL:Function_call_interrupts
     30063           -29.8%      21101 ±  6%  interrupts.CPU58.LOC:Local_timer_interrupts
    350.00 ± 26%     -81.4%      65.25 ± 46%  interrupts.CPU58.RES:Rescheduling_interrupts
      2401 ± 15%     -57.3%       1026 ± 13%  interrupts.CPU59.CAL:Function_call_interrupts
     30072           -29.8%      21111 ±  6%  interrupts.CPU59.LOC:Local_timer_interrupts
    276.75 ±  8%     -72.2%      77.00 ± 23%  interrupts.CPU59.RES:Rescheduling_interrupts
      2266 ± 15%     -53.0%       1065 ± 11%  interrupts.CPU6.CAL:Function_call_interrupts
     30063           -29.8%      21111 ±  6%  interrupts.CPU6.LOC:Local_timer_interrupts
    276.00 ± 10%     -65.0%      96.50 ± 37%  interrupts.CPU6.RES:Rescheduling_interrupts
      2667 ±  9%     -69.7%     808.25 ± 12%  interrupts.CPU60.CAL:Function_call_interrupts
     30061           -30.1%      21025 ±  6%  interrupts.CPU60.LOC:Local_timer_interrupts
    336.75 ± 15%     -82.2%      60.00 ± 38%  interrupts.CPU60.RES:Rescheduling_interrupts
      2366 ± 21%     -54.3%       1080 ± 18%  interrupts.CPU61.CAL:Function_call_interrupts
     30082           -29.8%      21105 ±  6%  interrupts.CPU61.LOC:Local_timer_interrupts
    311.00 ± 30%     -72.6%      85.25 ± 31%  interrupts.CPU61.RES:Rescheduling_interrupts
      2399 ± 18%     -64.3%     855.75 ±  7%  interrupts.CPU62.CAL:Function_call_interrupts
     30067           -30.1%      21030 ±  6%  interrupts.CPU62.LOC:Local_timer_interrupts
    284.00 ± 22%     -78.6%      60.75 ± 21%  interrupts.CPU62.RES:Rescheduling_interrupts
     30070           -29.8%      21103 ±  6%  interrupts.CPU63.LOC:Local_timer_interrupts
    382.00 ± 14%     -79.2%      79.50 ± 15%  interrupts.CPU63.RES:Rescheduling_interrupts
      2374 ± 26%     -63.2%     874.50 ±  6%  interrupts.CPU64.CAL:Function_call_interrupts
     30054           -29.9%      21079 ±  6%  interrupts.CPU64.LOC:Local_timer_interrupts
    299.75 ± 26%     -75.6%      73.25 ± 13%  interrupts.CPU64.RES:Rescheduling_interrupts
      2695 ± 24%     -65.7%     925.25 ±  7%  interrupts.CPU65.CAL:Function_call_interrupts
     30084           -30.0%      21044 ±  6%  interrupts.CPU65.LOC:Local_timer_interrupts
    358.50 ± 35%     -81.3%      67.00 ± 13%  interrupts.CPU65.RES:Rescheduling_interrupts
      2628 ± 18%     -61.7%       1006 ± 14%  interrupts.CPU66.CAL:Function_call_interrupts
     30078           -29.8%      21100 ±  6%  interrupts.CPU66.LOC:Local_timer_interrupts
    319.25 ± 21%     -72.3%      88.50 ± 14%  interrupts.CPU66.RES:Rescheduling_interrupts
     30056           -29.7%      21118 ±  6%  interrupts.CPU67.LOC:Local_timer_interrupts
    416.00 ± 31%     -80.5%      81.25 ±  8%  interrupts.CPU67.RES:Rescheduling_interrupts
      2677 ± 34%     -62.6%       1002 ± 17%  interrupts.CPU68.CAL:Function_call_interrupts
     30065           -29.2%      21297 ±  7%  interrupts.CPU68.LOC:Local_timer_interrupts
    343.50 ± 44%     -74.9%      86.25 ± 18%  interrupts.CPU68.RES:Rescheduling_interrupts
      2355 ± 32%     -64.9%     826.00 ± 18%  interrupts.CPU69.CAL:Function_call_interrupts
     30092           -29.9%      21094 ±  6%  interrupts.CPU69.LOC:Local_timer_interrupts
    296.50 ± 41%     -73.6%      78.25 ± 50%  interrupts.CPU69.RES:Rescheduling_interrupts
      2199 ± 31%     -54.4%       1003 ± 16%  interrupts.CPU7.CAL:Function_call_interrupts
     30062           -30.6%      20850 ±  3%  interrupts.CPU7.LOC:Local_timer_interrupts
    262.00 ± 38%     -72.9%      71.00 ± 29%  interrupts.CPU7.RES:Rescheduling_interrupts
      2467 ± 33%     -63.4%     902.50 ± 20%  interrupts.CPU70.CAL:Function_call_interrupts
     30062           -30.0%      21057 ±  6%  interrupts.CPU70.LOC:Local_timer_interrupts
    323.75 ± 39%     -76.1%      77.25 ± 36%  interrupts.CPU70.RES:Rescheduling_interrupts
      2268 ± 33%     -59.2%     925.00 ± 17%  interrupts.CPU71.CAL:Function_call_interrupts
     30060           -29.8%      21099 ±  6%  interrupts.CPU71.LOC:Local_timer_interrupts
    282.75 ± 38%     -76.0%      67.75 ± 24%  interrupts.CPU71.RES:Rescheduling_interrupts
     29888 ±  2%     -29.6%      21051 ±  6%  interrupts.CPU72.LOC:Local_timer_interrupts
     30063           -29.9%      21079 ±  6%  interrupts.CPU73.LOC:Local_timer_interrupts
     30072           -29.8%      21100 ±  6%  interrupts.CPU74.LOC:Local_timer_interrupts
     30034 ±  2%     -29.8%      21097 ±  6%  interrupts.CPU75.LOC:Local_timer_interrupts
     30027 ±  2%     -29.7%      21100 ±  6%  interrupts.CPU76.LOC:Local_timer_interrupts
     30038           -29.7%      21101 ±  6%  interrupts.CPU77.LOC:Local_timer_interrupts
     30065           -29.8%      21095 ±  6%  interrupts.CPU78.LOC:Local_timer_interrupts
     29971           -29.7%      21077 ±  6%  interrupts.CPU79.LOC:Local_timer_interrupts
      2632 ± 24%     -57.7%       1114 ± 17%  interrupts.CPU8.CAL:Function_call_interrupts
     30087           -30.2%      21001 ±  7%  interrupts.CPU8.LOC:Local_timer_interrupts
    337.00 ± 34%     -73.7%      88.75 ± 21%  interrupts.CPU8.RES:Rescheduling_interrupts
     30075           -30.0%      21051 ±  6%  interrupts.CPU80.LOC:Local_timer_interrupts
     30058           -30.0%      21049 ±  6%  interrupts.CPU81.LOC:Local_timer_interrupts
     30048           -29.9%      21067 ±  6%  interrupts.CPU82.LOC:Local_timer_interrupts
     30063           -30.1%      21023 ±  6%  interrupts.CPU83.LOC:Local_timer_interrupts
     30073 ±  2%     -30.0%      21055 ±  6%  interrupts.CPU84.LOC:Local_timer_interrupts
     30076           -30.0%      21050 ±  6%  interrupts.CPU85.LOC:Local_timer_interrupts
     30056           -30.0%      21053 ±  6%  interrupts.CPU86.LOC:Local_timer_interrupts
     30057           -29.9%      21080 ±  6%  interrupts.CPU87.LOC:Local_timer_interrupts
     29942           -29.6%      21071 ±  6%  interrupts.CPU88.LOC:Local_timer_interrupts
     30051           -30.0%      21028 ±  6%  interrupts.CPU89.LOC:Local_timer_interrupts
     30067           -29.8%      21113 ±  6%  interrupts.CPU9.LOC:Local_timer_interrupts
    324.50 ± 35%     -81.0%      61.75 ± 45%  interrupts.CPU9.RES:Rescheduling_interrupts
     30067 ±  2%     -29.9%      21084 ±  6%  interrupts.CPU90.LOC:Local_timer_interrupts
     30127 ±  3%     -30.3%      21006 ±  6%  interrupts.CPU91.LOC:Local_timer_interrupts
     29962 ±  2%     -29.6%      21091 ±  6%  interrupts.CPU92.LOC:Local_timer_interrupts
     30241           -30.0%      21181 ±  6%  interrupts.CPU93.LOC:Local_timer_interrupts
     30040           -29.6%      21159 ±  6%  interrupts.CPU94.LOC:Local_timer_interrupts
     30085           -29.9%      21096 ±  6%  interrupts.CPU95.LOC:Local_timer_interrupts
   2883576           -29.9%    2022083 ±  6%  interrupts.LOC:Local_timer_interrupts
     16309 ± 14%     -73.7%       4292 ±  3%  interrupts.RES:Rescheduling_interrupts


                                                                                
                              fsmark.time.system_time                           
                                                                                
  150 +---------------------------------------------------------------------+   
  140 |+++.+ ++  ++.++++ +++.+++ ++++.++++ ++.++++ +++.+++ ++++.++ +++++.+++|   
      |        ++                                                           |   
  130 |-+                                                                   |   
  120 |-+                                                                   |   
      |                                                                     |   
  110 |-+                                                                   |   
  100 |-+                                                                   |   
   90 |-+                                                                   |   
      |                                                                     |   
   80 |-+                                                                   |   
   70 |-+                                                                   |   
      |                       O OOO O  OOOOO     O OO    O                  |   
   60 |OOO OOOO OOO OOOO OOO O O   O  O     O OOO O  O OO                   |   
   50 +---------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                      fsmark.time.percent_of_cpu_this_job_got                   
                                                                                
  1100 +--------------------------------------------------------------------+   
  1050 |-+++  +++ ++   ++ ++  ++  ++++  +.+ ++  ++  ++  ++ ++   ++ ++  + +++|   
       |         +                                                      +   |   
  1000 |-+                                                                  |   
   950 |-+                                                                  |   
       |                                                                    |   
   900 |-+                                                                  |   
   850 |-+                                                                  |   
   800 |-+                                                                  |   
       |                                                                    |   
   750 |-+ O     O O O O          O                 O                       |   
   700 |OO   O O  O O     OOOOOO O OOOOO    OOOOOOO  O  O                   |   
       |  O   O O       OO              O OO          OO O                  |   
   650 |-+                                                                  |   
   600 +--------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                            fsmark.time.elapsed_time                            
                                                                                
  15 +----------------------------------------------------------------------+   
     |                                                                      |   
  14 |-++. +   ++.   + +     + + .++   +. +    +. +    +. +    +. + +  +.   |   
  13 |++  + +++   +++ + +.+++ + +   +++  + ++++  + ++++  + ++++  + + ++  +++|   
     |                                                                      |   
  12 |-+                                                                    |   
     |                                                                      |   
  11 |-+                                                                    |   
     |                                                                      |   
  10 |-+                                                                    |   
   9 |-+    O                          O OO      O   O   O                  |   
     |         OO    O    O  O O  OOOOO    O      OOO O                     |   
   8 |OOO OO O    OOO O O  OO O O           OOOO       O                    |   
     |        O        O                                                    |   
   7 +----------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                          fsmark.time.elapsed_time.max                          
                                                                                
  15 +----------------------------------------------------------------------+   
     |                                                                      |   
  14 |-++. +   ++.   + +     + + .++   +. +    +. +    +. +    +. + +  +.   |   
  13 |++  + +++   +++ + +.+++ + +   +++  + ++++  + ++++  + ++++  + + ++  +++|   
     |                                                                      |   
  12 |-+                                                                    |   
     |                                                                      |   
  11 |-+                                                                    |   
     |                                                                      |   
  10 |-+                                                                    |   
   9 |-+    O                          O OO      O   O   O                  |   
     |         OO    O    O  O O  OOOOO    O      OOO O                     |   
   8 |OOO OO O    OOO O O  OO O O           OOOO       O                    |   
     |        O        O                                                    |   
   7 +----------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                        fsmark.time.voluntary_context_switches                  
                                                                                
  2.6e+06 +-----------------------------------------------------------------+   
  2.4e+06 |++        + +      +      ++ +      +       +      +     .++  ++ |   
          | ++++++.++ + ++++++ ++++.+  + ++++++ +++.+++ ++++++ +++++   ++  +|   
  2.2e+06 |-+                                                               |   
    2e+06 |-+                                                               |   
          |                                                                 |   
  1.8e+06 |-+                                                               |   
  1.6e+06 |-+                                                               |   
  1.4e+06 |-+                                                               |   
          |                                                                 |   
  1.2e+06 |-+                                                               |   
    1e+06 |-+                                                               |   
          | OOO     O OOOO  OO OOOO OOOOOOOOOOOOOOO OOOOO O                 |   
   800000 |O+  OOO O O    O   O                          O                  |   
   600000 +-----------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                fsmark.files_per_sec                            
                                                                                
  16000 +-------------------------------------------------------------------+   
        |        O        O                                                 |   
  15000 |O+         O         O O                       O                   |   
  14000 |-OOOO  O OO OO O  O O    O          OOO O O                        |   
        |              O    O      O OOOO   O       O                       |   
  13000 |-+    O               O O        O       O  OOO O                  |   
  12000 |-+                              O O                                |   
        |                                                                   |   
  11000 |-+                                                                 |   
  10000 |-+                                                                 |   
        |                                                                   |   
   9000 |-+                                                                 |   
   8000 |+++++.++++++++++.++++++++++. ++++++++++.++++++++++.++++++++++.+++++|   
        |                            +                                      |   
   7000 +-------------------------------------------------------------------+   
                                                                                
                                                                                
[*] bisect-good sample
[O] bisect-bad  sample

***************************************************************************************************
lkp-csl-2sp7: 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 512G memory
=========================================================================================
compiler/cpufreq_governor/disk/filesize/fs/iterations/kconfig/nr_directories/nr_files_per_directory/nr_threads/rootfs/sync_method/tbox_group/test_size/testcase/ucode:
  gcc-9/performance/1SSD/8K/btrfs/1x/x86_64-rhel-8.3/16d/256fpd/32t/debian-10.4-x86_64-20200603.cgz/fsyncBeforeClose/lkp-csl-2sp7/400M/fsmark/0x5002f01

commit: 
  b010373b40 ("btrfs: fix potential overflow in cluster_pages_for_defrag on 32bit arch")
  fac2f60d5f ("btrfs: switch extent buffer tree lock to rw_semaphore")

b010373b4014116a fac2f60d5fe83fd45ee08a85c2e 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
    585190 ±  2%     -21.0%     462103 ±  5%  fsmark.app_overhead
      7968 ±  2%     +42.6%      11366 ±  3%  fsmark.files_per_sec
    584.75           -24.1%     444.00 ±  3%  fsmark.time.percent_of_cpu_this_job_got
     41.77           -44.5%      23.17 ±  2%  fsmark.time.system_time
    859174 ±  2%     -60.3%     340937 ±  8%  fsmark.time.voluntary_context_switches
    165800 ±  2%     -16.4%     138675 ±  4%  softirqs.RCU
    243885           -10.1%     219173        softirqs.SCHED
     87533 ±  5%     +24.8%     109270 ±  6%  vmstat.io.bo
    244338 ±  4%     -51.0%     119833 ±  8%  vmstat.system.cs
     91.10            +0.8%      91.86        iostat.cpu.idle
      0.97 ±  8%     +98.2%       1.92 ±  8%  iostat.cpu.iowait
      6.77 ±  3%     -24.9%       5.08 ±  4%  iostat.cpu.system
      1.19 ±  8%      +1.3        2.47 ±  7%  mpstat.cpu.all.iowait%
      0.89 ±  6%      +0.2        1.06 ±  6%  mpstat.cpu.all.irq%
      6.89            -2.2        4.69 ±  7%  mpstat.cpu.all.sys%
    233105 ±  6%     +13.3%     264135 ±  4%  numa-numastat.node0.numa_hit
    119912 ± 15%     -22.5%      92882 ±  7%  numa-numastat.node1.local_node
    135497 ±  9%     -23.3%     103865 ±  7%  numa-numastat.node1.numa_hit
     22327 ±  8%     -23.4%      17096 ± 10%  sched_debug.cfs_rq:/.min_vruntime.avg
     14173 ±  5%     -22.4%      10993 ± 13%  sched_debug.cfs_rq:/.min_vruntime.min
    -15.50           +35.5%     -21.00        sched_debug.cpu.nr_uninterruptible.min
      1956            -9.4%       1772 ±  2%  meminfo.Active(anon)
  21216768 ±  5%      +9.9%   23318528 ±  4%  meminfo.DirectMap2M
     11775 ±  7%     -39.4%       7137 ± 17%  meminfo.Dirty
    369943           +25.6%     464774 ±  5%  meminfo.max_used_kB
    488.75            -9.4%     443.00 ±  2%  proc-vmstat.nr_active_anon
      2913 ±  6%     -39.7%       1757 ± 16%  proc-vmstat.nr_dirty
      2845            -1.3%       2807        proc-vmstat.nr_shmem
    488.75            -9.4%     443.00 ±  2%  proc-vmstat.nr_zone_active_anon
      1955 ±  3%     -17.3%       1617 ±  4%  proc-vmstat.nr_zone_write_pending
  63044510 ±  2%     -82.2%   11239499 ±  4%  cpuidle.C1.time
    578945           -78.4%     125019 ±  2%  cpuidle.C1.usage
 6.614e+08 ±  5%     -12.0%  5.817e+08 ±  4%  cpuidle.C1E.time
   2104719 ±  2%     -26.4%    1549637        cpuidle.C1E.usage
    247244           -46.9%     131378 ±  3%  cpuidle.POLL.time
     62458 ±  4%     -43.5%      35316 ±  8%  cpuidle.POLL.usage
    579.00 ± 10%     -45.1%     318.00 ± 20%  slabinfo.biovec-128.active_objs
    579.00 ± 10%     -45.1%     318.00 ± 20%  slabinfo.biovec-128.num_objs
      2083 ±  3%     -29.5%       1467 ± 16%  slabinfo.biovec-64.active_objs
      2083 ±  3%     -29.5%       1467 ± 16%  slabinfo.biovec-64.num_objs
      4038           -11.0%       3595 ±  5%  slabinfo.dmaengine-unmap-16.active_objs
      4038           -10.9%       3597 ±  4%  slabinfo.dmaengine-unmap-16.num_objs
      9642 ±  3%     -12.6%       8424 ±  7%  slabinfo.fsnotify_mark_connector.active_objs
      9642 ±  3%     -12.6%       8424 ±  7%  slabinfo.fsnotify_mark_connector.num_objs
      6201 ±  2%     -97.5%     155.00 ± 20%  slabinfo.numa_policy.active_objs
      6201 ±  2%     -97.5%     155.00 ± 20%  slabinfo.numa_policy.num_objs
      2176 ±  7%     -21.9%       1700 ±  9%  slabinfo.skbuff_fclone_cache.active_objs
      2176 ±  7%     -21.9%       1700 ±  9%  slabinfo.skbuff_fclone_cache.num_objs
    154478 ± 37%     +59.9%     246991 ±  4%  numa-meminfo.node0.AnonPages
      8598 ±  7%     -20.6%       6824 ±  7%  numa-meminfo.node0.Dirty
    323193 ± 13%     +41.1%     456134 ±  6%  numa-meminfo.node0.Inactive
    161361 ± 36%     +53.6%     247897 ±  4%  numa-meminfo.node0.Inactive(anon)
    161831 ± 14%     +28.7%     208236 ±  9%  numa-meminfo.node0.Inactive(file)
     18148 ± 12%     -22.1%      14136        numa-meminfo.node0.Mapped
      7836 ± 38%     +45.0%      11362 ±  7%  numa-meminfo.node0.PageTables
    119539 ±  5%      +7.9%     128962 ±  2%  numa-meminfo.node0.SUnreclaim
     37998 ±  8%     -74.1%       9848 ± 31%  numa-meminfo.node1.Active
     37186 ±  9%     -76.0%       8910 ± 39%  numa-meminfo.node1.Active(file)
      3056 ± 27%     -81.3%     570.50 ± 54%  numa-meminfo.node1.Dirty
    192203 ± 32%     -52.5%      91270 ±  9%  numa-meminfo.node1.Inactive
     49142 ± 15%     -69.5%      15004 ± 52%  numa-meminfo.node1.Inactive(file)
     15411 ± 14%     +26.5%      19495        numa-meminfo.node1.Mapped
   1263446 ±  8%     -18.8%    1025804 ±  4%  numa-meminfo.node1.MemUsed
     90893 ±  6%     -13.8%      78384 ±  3%  numa-meminfo.node1.SUnreclaim
      3304 ±107%    +180.9%       9281 ±  2%  numa-meminfo.node1.Shmem
    135769 ±  5%     -12.1%     119352 ±  9%  numa-meminfo.node1.Slab
     24515 ± 17%     +29.3%      31692 ± 12%  numa-vmstat.node0.nr_active_file
     38656 ± 37%     +61.3%      62334 ±  4%  numa-vmstat.node0.nr_anon_pages
     73512 ±  5%     +23.7%      90938 ±  8%  numa-vmstat.node0.nr_dirtied
      2124 ±  8%     -17.4%       1754 ± 11%  numa-vmstat.node0.nr_dirty
     40379 ± 36%     +54.9%      62556 ±  4%  numa-vmstat.node0.nr_inactive_anon
     40038 ± 14%     +34.0%      53639 ±  6%  numa-vmstat.node0.nr_inactive_file
      1959 ± 38%     +48.1%       2900 ±  4%  numa-vmstat.node0.nr_page_table_pages
     29852 ±  5%      +8.2%      32297 ±  2%  numa-vmstat.node0.nr_slab_unreclaimable
     71959 ±  5%     +24.2%      89408 ±  8%  numa-vmstat.node0.nr_written
     24515 ± 17%     +29.3%      31687 ± 12%  numa-vmstat.node0.nr_zone_active_file
     40379 ± 36%     +54.9%      62556 ±  4%  numa-vmstat.node0.nr_zone_inactive_anon
     40041 ± 14%     +34.0%      53639 ±  6%  numa-vmstat.node0.nr_zone_inactive_file
      9129 ±  9%     -74.1%       2360 ± 43%  numa-vmstat.node1.nr_active_file
     25041 ± 20%     -68.4%       7908 ± 41%  numa-vmstat.node1.nr_dirtied
    775.25 ± 26%     -82.3%     137.50 ± 53%  numa-vmstat.node1.nr_dirty
     12257 ± 14%     -67.3%       4010 ± 58%  numa-vmstat.node1.nr_inactive_file
      3919 ± 15%     +27.1%       4983 ±  2%  numa-vmstat.node1.nr_mapped
    826.00 ±107%    +180.8%       2319 ±  2%  numa-vmstat.node1.nr_shmem
     22725 ±  6%     -13.9%      19568 ±  3%  numa-vmstat.node1.nr_slab_unreclaimable
     24390 ± 19%     -67.9%       7832 ± 40%  numa-vmstat.node1.nr_written
      9129 ±  9%     -74.1%       2360 ± 43%  numa-vmstat.node1.nr_zone_active_file
     12257 ± 14%     -67.3%       4010 ± 58%  numa-vmstat.node1.nr_zone_inactive_file
    390.75 ± 32%     -80.5%      76.25 ± 92%  numa-vmstat.node1.nr_zone_write_pending
     10.85 ±143%     -10.8        0.00        perf-profile.calltrace.cycles-pp.read
      4.59 ±112%      -4.6        0.00        perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read
      4.59 ±112%      -4.6        0.00        perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
      4.59 ±112%      -4.6        0.00        perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
      4.59 ±112%      -4.6        0.00        perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
      3.74 ±104%      -3.7        0.00        perf-profile.calltrace.cycles-pp.bprm_execve.do_execveat_common.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe
      3.74 ±104%      -3.7        0.00        perf-profile.calltrace.cycles-pp.exec_binprm.bprm_execve.do_execveat_common.__x64_sys_execve.do_syscall_64
      3.74 ±104%      -3.7        0.00        perf-profile.calltrace.cycles-pp.load_elf_binary.exec_binprm.bprm_execve.do_execveat_common.__x64_sys_execve
      4.59 ±112%      -2.3        2.27 ±173%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__lxstat64
      4.59 ±112%      -2.3        2.27 ±173%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__lxstat64
      4.59 ±112%      -2.3        2.27 ±173%  perf-profile.calltrace.cycles-pp.link_path_walk.path_lookupat.filename_lookup.vfs_statx.__do_sys_newlstat
      4.59 ±112%      -2.3        2.27 ±173%  perf-profile.calltrace.cycles-pp.__do_sys_newlstat.do_syscall_64.entry_SYSCALL_64_after_hwframe.__lxstat64
      4.59 ±112%      -2.3        2.27 ±173%  perf-profile.calltrace.cycles-pp.page_remove_rmap.zap_pte_range.unmap_page_range.unmap_vmas.exit_mmap
      4.59 ±112%      -2.3        2.27 ±173%  perf-profile.calltrace.cycles-pp.filename_lookup.vfs_statx.__do_sys_newlstat.do_syscall_64.entry_SYSCALL_64_after_hwframe
      4.59 ±112%      -2.3        2.27 ±173%  perf-profile.calltrace.cycles-pp.path_lookupat.filename_lookup.vfs_statx.__do_sys_newlstat.do_syscall_64
      4.59 ±112%      -2.3        2.27 ±173%  perf-profile.calltrace.cycles-pp.vfs_statx.__do_sys_newlstat.do_syscall_64.entry_SYSCALL_64_after_hwframe.__lxstat64
      4.59 ±112%      -2.3        2.27 ±173%  perf-profile.calltrace.cycles-pp.__lxstat64
      4.59 ±112%      +0.8        5.36 ±173%  perf-profile.calltrace.cycles-pp.show_interrupts.seq_read.proc_reg_read.vfs_read.ksys_read
      7.72 ±132%      -7.7        0.00        perf-profile.children.cycles-pp.read
      3.74 ±104%      -3.7        0.00        perf-profile.children.cycles-pp.__x64_sys_execve
      3.74 ±104%      -3.7        0.00        perf-profile.children.cycles-pp.do_execveat_common
      3.74 ±104%      -3.7        0.00        perf-profile.children.cycles-pp.bprm_execve
      3.74 ±104%      -3.7        0.00        perf-profile.children.cycles-pp.exec_binprm
      3.74 ±104%      -3.7        0.00        perf-profile.children.cycles-pp.load_elf_binary
      4.59 ±112%      -3.2        1.39 ±173%  perf-profile.children.cycles-pp.lockref_get_not_dead
      4.59 ±112%      -2.8        1.78 ±173%  perf-profile.children.cycles-pp.__legitimize_path
      4.59 ±112%      -2.8        1.78 ±173%  perf-profile.children.cycles-pp.unlazy_walk
      4.59 ±112%      -2.3        2.27 ±173%  perf-profile.children.cycles-pp.__do_sys_newlstat
      4.59 ±112%      -2.3        2.27 ±173%  perf-profile.children.cycles-pp.page_remove_rmap
      4.59 ±112%      -2.3        2.27 ±173%  perf-profile.children.cycles-pp.__lxstat64
      4.59 ±112%      +0.8        5.36 ±173%  perf-profile.children.cycles-pp.show_interrupts
      4.59 ±112%      -3.2        1.39 ±173%  perf-profile.self.cycles-pp.lockref_get_not_dead
      4.59 ±112%      -2.3        2.27 ±173%  perf-profile.self.cycles-pp.page_remove_rmap
     10.74           -20.1%       8.58 ±  4%  perf-stat.i.MPKI
      2.98            -0.2        2.74 ±  6%  perf-stat.i.branch-miss-rate%
  18754694 ± 16%     -41.8%   10910993 ± 29%  perf-stat.i.cache-misses
 1.258e+08           -14.1%  1.081e+08 ±  2%  perf-stat.i.cache-references
    324941           -48.6%     167177 ±  3%  perf-stat.i.context-switches
      2.39           -27.7%       1.73 ±  2%  perf-stat.i.cpi
 2.823e+10 ±  2%     -23.4%  2.163e+10        perf-stat.i.cpu-cycles
      1598 ± 35%     -54.5%     727.08 ± 27%  perf-stat.i.cpu-migrations
      0.03 ±  7%      -0.0        0.02 ±  4%  perf-stat.i.dTLB-load-miss-rate%
    118457 ±  4%     +12.9%     133779 ±  5%  perf-stat.i.dTLB-store-misses
     38.10            +1.5       39.61        perf-stat.i.iTLB-load-miss-rate%
   4290155           -11.4%    3800208        perf-stat.i.iTLB-load-misses
   7498866 ±  2%     -15.3%    6354604        perf-stat.i.iTLB-loads
      2912           +16.9%       3404 ±  7%  perf-stat.i.instructions-per-iTLB-miss
      0.48           +32.8%       0.64 ±  5%  perf-stat.i.ipc
      0.29 ±  2%     -23.5%       0.22        perf-stat.i.metric.GHz
     12172 ±  5%     +27.6%      15531 ±  5%  perf-stat.i.minor-faults
     86.54 ±  2%     -10.2       76.32 ±  8%  perf-stat.i.node-load-miss-rate%
   5533201 ± 15%     -63.4%    2023764 ± 47%  perf-stat.i.node-load-misses
    569588 ±  8%     -22.4%     442256 ± 15%  perf-stat.i.node-loads
     73.07 ±  6%     -23.8       49.27 ± 20%  perf-stat.i.node-store-miss-rate%
   1955072 ± 16%     -61.9%     744999 ± 45%  perf-stat.i.node-store-misses
    469220           +27.9%     600128        perf-stat.i.node-stores
     12173 ±  5%     +27.6%      15532 ±  5%  perf-stat.i.page-faults
     10.42           -18.6%       8.49 ±  7%  perf-stat.overall.MPKI
      3.08            -0.3        2.81 ±  8%  perf-stat.overall.branch-miss-rate%
     14.88 ± 15%      -4.8       10.11 ± 30%  perf-stat.overall.cache-miss-rate%
      2.34           -27.5%       1.70 ±  5%  perf-stat.overall.cpi
      0.03 ±  6%      -0.0        0.03 ±  5%  perf-stat.overall.dTLB-load-miss-rate%
     36.39            +1.0       37.41        perf-stat.overall.iTLB-load-miss-rate%
      2814           +19.8%       3371 ±  7%  perf-stat.overall.instructions-per-iTLB-miss
      0.43           +38.3%       0.59 ±  5%  perf-stat.overall.ipc
     90.59           -11.3       79.27 ±  9%  perf-stat.overall.node-load-miss-rate%
     80.31 ±  3%     -28.0       52.32 ± 22%  perf-stat.overall.node-store-miss-rate%
  68514320 ±  2%     -10.3%   61424359 ± 13%  perf-stat.ps.branch-misses
  16421948 ± 16%     -44.8%    9057139 ± 29%  perf-stat.ps.cache-misses
 1.102e+08           -18.5%   89825260 ±  2%  perf-stat.ps.cache-references
    284527           -51.1%     139009 ±  3%  perf-stat.ps.context-switches
     84053            -5.0%      79818        perf-stat.ps.cpu-clock
 2.472e+10 ±  2%     -27.3%  1.797e+10        perf-stat.ps.cpu-cycles
      1396 ± 35%     -56.8%     602.64 ± 27%  perf-stat.ps.cpu-migrations
    725466 ±  6%     -15.4%     614094 ±  3%  perf-stat.ps.dTLB-load-misses
   3756884           -16.0%    3156866        perf-stat.ps.iTLB-load-misses
   6567976 ±  2%     -19.6%    5281681        perf-stat.ps.iTLB-loads
     10638 ±  5%     +21.0%      12869 ±  5%  perf-stat.ps.minor-faults
   4846606 ± 15%     -65.3%    1680351 ± 47%  perf-stat.ps.node-load-misses
    498627 ±  8%     -26.4%     367024 ± 15%  perf-stat.ps.node-loads
   1712364 ± 16%     -63.9%     618693 ± 45%  perf-stat.ps.node-store-misses
    410739           +21.4%     498452 ±  2%  perf-stat.ps.node-stores
     10639 ±  5%     +21.0%      12870 ±  5%  perf-stat.ps.page-faults
     84053            -5.0%      79818        perf-stat.ps.task-clock
 8.562e+10           -24.3%  6.483e+10 ±  5%  perf-stat.total.instructions
     89183           -30.0%      62464        interrupts.CAL:Function_call_interrupts
     17434           -22.0%      13594        interrupts.CPU0.LOC:Local_timer_interrupts
     96.75 ± 32%     -57.9%      40.75 ± 28%  interrupts.CPU0.RES:Rescheduling_interrupts
      1709 ± 19%     -36.9%       1079 ± 31%  interrupts.CPU1.CAL:Function_call_interrupts
     17557           -22.3%      13648        interrupts.CPU1.LOC:Local_timer_interrupts
    121.50 ± 11%     -53.7%      56.25 ± 31%  interrupts.CPU1.RES:Rescheduling_interrupts
      1102 ± 13%     -25.6%     820.50 ±  5%  interrupts.CPU10.CAL:Function_call_interrupts
     17474           -21.9%      13650        interrupts.CPU10.LOC:Local_timer_interrupts
    110.50 ± 20%     -54.5%      50.25 ± 12%  interrupts.CPU10.RES:Rescheduling_interrupts
      1113 ±  3%     -41.7%     648.50 ± 12%  interrupts.CPU11.CAL:Function_call_interrupts
     17488           -23.1%      13448 ±  2%  interrupts.CPU11.LOC:Local_timer_interrupts
    124.50 ±  7%     -71.3%      35.75 ± 42%  interrupts.CPU11.RES:Rescheduling_interrupts
      1050 ±  7%     -30.9%     725.75 ± 12%  interrupts.CPU12.CAL:Function_call_interrupts
     17517           -22.9%      13502        interrupts.CPU12.LOC:Local_timer_interrupts
    107.25 ± 10%     -63.4%      39.25 ± 20%  interrupts.CPU12.RES:Rescheduling_interrupts
      1171 ±  6%     -36.0%     749.75 ±  8%  interrupts.CPU13.CAL:Function_call_interrupts
     17718 ±  2%     -22.7%      13705        interrupts.CPU13.LOC:Local_timer_interrupts
    123.00 ± 10%     -61.2%      47.75 ± 14%  interrupts.CPU13.RES:Rescheduling_interrupts
     17545           -22.3%      13627        interrupts.CPU14.LOC:Local_timer_interrupts
      1100 ±  4%     -36.6%     698.00 ± 14%  interrupts.CPU15.CAL:Function_call_interrupts
     17505           -20.6%      13896        interrupts.CPU15.LOC:Local_timer_interrupts
    105.75 ± 22%     -57.9%      44.50 ± 41%  interrupts.CPU15.RES:Rescheduling_interrupts
      1162 ±  7%     -36.0%     743.25 ± 13%  interrupts.CPU16.CAL:Function_call_interrupts
     17767 ±  2%     -23.3%      13622        interrupts.CPU16.LOC:Local_timer_interrupts
    652.00 ± 82%     -93.1%      45.00 ± 18%  interrupts.CPU16.RES:Rescheduling_interrupts
      1093 ± 13%     -41.6%     638.50 ± 18%  interrupts.CPU17.CAL:Function_call_interrupts
     17451           -22.0%      13608        interrupts.CPU17.LOC:Local_timer_interrupts
      1244 ±  6%     -40.1%     745.25 ± 24%  interrupts.CPU18.CAL:Function_call_interrupts
     17679           -23.2%      13575        interrupts.CPU18.LOC:Local_timer_interrupts
      1114 ±  3%     -27.7%     805.50 ±  7%  interrupts.CPU19.CAL:Function_call_interrupts
     17403           -21.8%      13616        interrupts.CPU19.LOC:Local_timer_interrupts
    132.75 ± 20%     -66.9%      44.00 ± 11%  interrupts.CPU19.RES:Rescheduling_interrupts
      1202 ±  6%     -35.8%     771.75 ±  5%  interrupts.CPU2.CAL:Function_call_interrupts
     17563           -22.3%      13651        interrupts.CPU2.LOC:Local_timer_interrupts
    124.00 ± 23%     -61.7%      47.50 ± 17%  interrupts.CPU2.RES:Rescheduling_interrupts
     17522           -22.7%      13543        interrupts.CPU20.LOC:Local_timer_interrupts
    380.75 ±112%     -88.1%      45.25 ± 19%  interrupts.CPU20.RES:Rescheduling_interrupts
      1180 ±  5%     -41.3%     692.50 ±  9%  interrupts.CPU21.CAL:Function_call_interrupts
     17518           -22.3%      13603        interrupts.CPU21.LOC:Local_timer_interrupts
    133.50 ± 24%     -68.2%      42.50 ± 20%  interrupts.CPU21.RES:Rescheduling_interrupts
      1158 ±  7%     -36.3%     738.00 ±  3%  interrupts.CPU22.CAL:Function_call_interrupts
     17488           -23.5%      13387        interrupts.CPU22.LOC:Local_timer_interrupts
    126.75 ± 26%     -69.4%      38.75 ± 11%  interrupts.CPU22.RES:Rescheduling_interrupts
     17535           -22.5%      13593        interrupts.CPU23.LOC:Local_timer_interrupts
    115.50 ± 29%     -70.8%      33.75 ±  3%  interrupts.CPU23.RES:Rescheduling_interrupts
    661.50 ± 12%     -29.1%     469.00 ± 10%  interrupts.CPU24.CAL:Function_call_interrupts
     17519           -22.6%      13552        interrupts.CPU24.LOC:Local_timer_interrupts
    622.00 ±  9%     -36.1%     397.25 ± 26%  interrupts.CPU25.CAL:Function_call_interrupts
     17543           -22.3%      13632        interrupts.CPU25.LOC:Local_timer_interrupts
    791.00 ± 34%     -43.6%     446.50 ±  8%  interrupts.CPU26.CAL:Function_call_interrupts
     17517           -23.1%      13471 ±  3%  interrupts.CPU26.LOC:Local_timer_interrupts
    644.25 ±  5%     -38.5%     396.00 ± 18%  interrupts.CPU27.CAL:Function_call_interrupts
     17521           -22.3%      13622        interrupts.CPU27.LOC:Local_timer_interrupts
    672.75 ± 12%     -31.4%     461.25 ± 10%  interrupts.CPU28.CAL:Function_call_interrupts
     17521           -22.5%      13579        interrupts.CPU28.LOC:Local_timer_interrupts
     17467           -22.2%      13593        interrupts.CPU29.LOC:Local_timer_interrupts
      1116 ± 11%     -35.0%     726.25 ± 16%  interrupts.CPU3.CAL:Function_call_interrupts
     17481           -22.8%      13500        interrupts.CPU3.LOC:Local_timer_interrupts
    113.25 ± 28%     -68.2%      36.00 ± 28%  interrupts.CPU3.RES:Rescheduling_interrupts
    688.00 ±  5%     -35.9%     441.00 ±  3%  interrupts.CPU30.CAL:Function_call_interrupts
     17528           -23.0%      13495        interrupts.CPU30.LOC:Local_timer_interrupts
    685.75 ±  7%     -36.0%     439.00 ±  3%  interrupts.CPU31.CAL:Function_call_interrupts
     17519           -22.3%      13618        interrupts.CPU31.LOC:Local_timer_interrupts
     17531           -22.4%      13611        interrupts.CPU32.LOC:Local_timer_interrupts
    690.25 ± 13%     -33.3%     460.50 ±  7%  interrupts.CPU33.CAL:Function_call_interrupts
     17535           -22.4%      13607        interrupts.CPU33.LOC:Local_timer_interrupts
     17412           -21.8%      13620        interrupts.CPU34.LOC:Local_timer_interrupts
     17518           -22.9%      13512        interrupts.CPU35.LOC:Local_timer_interrupts
    646.25 ±  7%     -28.9%     459.75 ±  6%  interrupts.CPU36.CAL:Function_call_interrupts
     17526           -22.3%      13624        interrupts.CPU36.LOC:Local_timer_interrupts
    679.00 ± 13%     -33.9%     448.50 ±  3%  interrupts.CPU37.CAL:Function_call_interrupts
     17259 ±  3%     -21.3%      13578        interrupts.CPU37.LOC:Local_timer_interrupts
    638.25 ±  6%     -30.7%     442.50 ±  2%  interrupts.CPU38.CAL:Function_call_interrupts
     17205 ±  3%     -20.9%      13605        interrupts.CPU38.LOC:Local_timer_interrupts
    732.50 ±  3%     -39.4%     443.75        interrupts.CPU39.CAL:Function_call_interrupts
     17534           -22.4%      13600        interrupts.CPU39.LOC:Local_timer_interrupts
      1130 ±  6%     -34.9%     735.25 ± 13%  interrupts.CPU4.CAL:Function_call_interrupts
     17492           -21.9%      13653        interrupts.CPU4.LOC:Local_timer_interrupts
    123.00 ± 25%     -65.9%      42.00 ± 39%  interrupts.CPU4.RES:Rescheduling_interrupts
     17575           -23.4%      13467        interrupts.CPU40.LOC:Local_timer_interrupts
    666.00 ± 10%     -28.5%     476.00 ±  6%  interrupts.CPU41.CAL:Function_call_interrupts
     17517           -22.8%      13522 ±  2%  interrupts.CPU41.LOC:Local_timer_interrupts
    627.25 ± 12%     -26.5%     461.25 ±  7%  interrupts.CPU42.CAL:Function_call_interrupts
     17525           -23.7%      13379 ±  4%  interrupts.CPU42.LOC:Local_timer_interrupts
     17519           -22.5%      13573        interrupts.CPU43.LOC:Local_timer_interrupts
    678.75 ±  9%     -30.4%     472.25 ±  8%  interrupts.CPU44.CAL:Function_call_interrupts
     17536           -22.2%      13640        interrupts.CPU44.LOC:Local_timer_interrupts
    714.50 ± 11%     -35.1%     464.00 ±  7%  interrupts.CPU45.CAL:Function_call_interrupts
     17533           -22.5%      13588        interrupts.CPU45.LOC:Local_timer_interrupts
    695.25 ±  3%     -34.6%     455.00 ±  5%  interrupts.CPU46.CAL:Function_call_interrupts
     17514           -22.4%      13599        interrupts.CPU46.LOC:Local_timer_interrupts
    718.25 ± 15%     -36.6%     455.25 ±  5%  interrupts.CPU47.CAL:Function_call_interrupts
     17530           -22.3%      13623        interrupts.CPU47.LOC:Local_timer_interrupts
      1091 ±  6%     -39.5%     660.25 ± 16%  interrupts.CPU48.CAL:Function_call_interrupts
     17501           -22.5%      13565        interrupts.CPU48.LOC:Local_timer_interrupts
    109.25 ± 24%     -69.8%      33.00 ± 39%  interrupts.CPU48.RES:Rescheduling_interrupts
      1097 ±  2%     -33.8%     726.25 ±  4%  interrupts.CPU49.CAL:Function_call_interrupts
     17494           -22.3%      13596        interrupts.CPU49.LOC:Local_timer_interrupts
    128.50 ± 21%     -66.9%      42.50 ±  9%  interrupts.CPU49.RES:Rescheduling_interrupts
      1078 ± 10%     -35.3%     697.50 ± 14%  interrupts.CPU5.CAL:Function_call_interrupts
     17473           -21.0%      13795        interrupts.CPU5.LOC:Local_timer_interrupts
    100.50 ± 23%     -49.5%      50.75 ± 54%  interrupts.CPU5.RES:Rescheduling_interrupts
      1093 ±  6%     -33.1%     731.50 ± 10%  interrupts.CPU50.CAL:Function_call_interrupts
     17535           -22.3%      13623        interrupts.CPU50.LOC:Local_timer_interrupts
    120.50 ± 31%     -69.3%      37.00 ± 20%  interrupts.CPU50.RES:Rescheduling_interrupts
      1596 ± 63%     -51.2%     779.25 ±  7%  interrupts.CPU51.CAL:Function_call_interrupts
     17519           -22.2%      13638        interrupts.CPU51.LOC:Local_timer_interrupts
    131.75 ± 26%     -55.0%      59.25 ± 30%  interrupts.CPU51.RES:Rescheduling_interrupts
      1006 ±  7%     -22.5%     780.50 ±  4%  interrupts.CPU52.CAL:Function_call_interrupts
     17485           -22.1%      13620        interrupts.CPU52.LOC:Local_timer_interrupts
    113.75 ± 14%     -56.3%      49.75 ± 20%  interrupts.CPU52.RES:Rescheduling_interrupts
     17534           -22.4%      13610        interrupts.CPU53.LOC:Local_timer_interrupts
    107.50 ± 32%     -53.7%      49.75 ± 10%  interrupts.CPU53.RES:Rescheduling_interrupts
     17461           -22.0%      13627        interrupts.CPU54.LOC:Local_timer_interrupts
     96.50 ± 24%     -59.1%      39.50 ± 23%  interrupts.CPU54.RES:Rescheduling_interrupts
      1068 ± 15%     -23.8%     814.00 ±  6%  interrupts.CPU55.CAL:Function_call_interrupts
     17544           -22.4%      13620        interrupts.CPU55.LOC:Local_timer_interrupts
     17422           -21.8%      13628        interrupts.CPU56.LOC:Local_timer_interrupts
     97.25 ± 13%     -51.9%      46.75 ± 26%  interrupts.CPU56.RES:Rescheduling_interrupts
      1043 ±  2%     -25.9%     772.75 ± 12%  interrupts.CPU57.CAL:Function_call_interrupts
     17517           -22.2%      13624        interrupts.CPU57.LOC:Local_timer_interrupts
    107.50 ± 18%     -56.5%      46.75 ± 21%  interrupts.CPU57.RES:Rescheduling_interrupts
    985.50 ±  3%     -21.0%     778.25 ±  5%  interrupts.CPU58.CAL:Function_call_interrupts
     17535           -22.1%      13652        interrupts.CPU58.LOC:Local_timer_interrupts
    101.00 ± 11%     -60.1%      40.25 ± 16%  interrupts.CPU58.RES:Rescheduling_interrupts
      1135 ±  2%     -35.1%     736.50 ±  6%  interrupts.CPU59.CAL:Function_call_interrupts
     17537           -22.2%      13644        interrupts.CPU59.LOC:Local_timer_interrupts
    105.25 ±  7%     -58.2%      44.00 ± 21%  interrupts.CPU59.RES:Rescheduling_interrupts
      1130 ±  5%     -31.5%     774.00 ±  3%  interrupts.CPU6.CAL:Function_call_interrupts
     17535           -22.1%      13659        interrupts.CPU6.LOC:Local_timer_interrupts
    125.25 ± 12%     -66.7%      41.75 ± 30%  interrupts.CPU6.RES:Rescheduling_interrupts
      1143 ± 10%     -33.0%     766.25 ±  9%  interrupts.CPU60.CAL:Function_call_interrupts
     17378           -21.6%      13619        interrupts.CPU60.LOC:Local_timer_interrupts
    120.00 ± 20%     -61.2%      46.50 ± 15%  interrupts.CPU60.RES:Rescheduling_interrupts
      1149 ±  7%     -36.6%     729.25 ± 10%  interrupts.CPU61.CAL:Function_call_interrupts
     17382           -22.0%      13557        interrupts.CPU61.LOC:Local_timer_interrupts
    125.00 ± 22%     -65.4%      43.25 ± 17%  interrupts.CPU61.RES:Rescheduling_interrupts
      1218 ±  8%     -32.0%     828.75 ± 18%  interrupts.CPU62.CAL:Function_call_interrupts
     17561           -22.4%      13626        interrupts.CPU62.LOC:Local_timer_interrupts
    130.75 ± 15%     -57.9%      55.00 ± 30%  interrupts.CPU62.RES:Rescheduling_interrupts
      1266 ±  8%     -35.1%     822.00 ±  3%  interrupts.CPU63.CAL:Function_call_interrupts
     17551           -22.4%      13617        interrupts.CPU63.LOC:Local_timer_interrupts
    134.75 ± 14%     -65.5%      46.50 ± 19%  interrupts.CPU63.RES:Rescheduling_interrupts
      1375 ±  9%     -43.8%     773.00 ±  9%  interrupts.CPU64.CAL:Function_call_interrupts
     17547           -23.4%      13440 ±  2%  interrupts.CPU64.LOC:Local_timer_interrupts
    149.50 ± 20%     -68.2%      47.50 ± 24%  interrupts.CPU64.RES:Rescheduling_interrupts
     17416           -21.6%      13652        interrupts.CPU65.LOC:Local_timer_interrupts
    136.75 ± 25%     -56.3%      59.75 ± 24%  interrupts.CPU65.RES:Rescheduling_interrupts
      1297 ±  5%     -35.6%     835.75 ±  8%  interrupts.CPU66.CAL:Function_call_interrupts
     17492           -22.0%      13637        interrupts.CPU66.LOC:Local_timer_interrupts
    138.50 ± 14%     -68.4%      43.75 ± 25%  interrupts.CPU66.RES:Rescheduling_interrupts
      1185 ± 10%     -36.6%     751.50 ± 10%  interrupts.CPU67.CAL:Function_call_interrupts
     17523           -22.4%      13592        interrupts.CPU67.LOC:Local_timer_interrupts
    122.75 ± 21%     -65.4%      42.50 ± 37%  interrupts.CPU67.RES:Rescheduling_interrupts
      1248 ±  7%     -37.7%     777.00 ±  6%  interrupts.CPU68.CAL:Function_call_interrupts
     17491           -22.4%      13565        interrupts.CPU68.LOC:Local_timer_interrupts
    110.00 ± 25%     -58.9%      45.25 ± 21%  interrupts.CPU68.RES:Rescheduling_interrupts
      1134 ±  6%     -29.1%     804.00 ± 16%  interrupts.CPU69.CAL:Function_call_interrupts
     17535           -22.6%      13578        interrupts.CPU69.LOC:Local_timer_interrupts
    106.25 ± 10%     -61.9%      40.50 ± 33%  interrupts.CPU69.RES:Rescheduling_interrupts
      1756 ± 59%     -55.1%     789.00 ± 10%  interrupts.CPU7.CAL:Function_call_interrupts
     17543           -22.4%      13622        interrupts.CPU7.LOC:Local_timer_interrupts
    123.00 ±  5%     -59.8%      49.50 ± 16%  interrupts.CPU7.RES:Rescheduling_interrupts
      1099 ± 10%     -33.5%     731.25 ±  6%  interrupts.CPU70.CAL:Function_call_interrupts
     17492           -22.1%      13621        interrupts.CPU70.LOC:Local_timer_interrupts
    109.25 ± 24%     -64.3%      39.00 ± 21%  interrupts.CPU70.RES:Rescheduling_interrupts
      1095 ±  6%     -33.5%     728.00 ±  5%  interrupts.CPU71.CAL:Function_call_interrupts
     17525           -22.7%      13540        interrupts.CPU71.LOC:Local_timer_interrupts
    115.50 ± 28%     -63.9%      41.75 ± 14%  interrupts.CPU71.RES:Rescheduling_interrupts
    676.50 ± 11%     -33.6%     449.50 ±  4%  interrupts.CPU72.CAL:Function_call_interrupts
     17515           -22.3%      13604        interrupts.CPU72.LOC:Local_timer_interrupts
    659.25 ±  7%     -31.9%     449.00 ±  5%  interrupts.CPU73.CAL:Function_call_interrupts
     17533           -21.3%      13801 ±  3%  interrupts.CPU73.LOC:Local_timer_interrupts
    696.00 ± 16%     -33.5%     463.00 ±  6%  interrupts.CPU74.CAL:Function_call_interrupts
     17535           -22.4%      13604        interrupts.CPU74.LOC:Local_timer_interrupts
     17534           -22.3%      13630        interrupts.CPU75.LOC:Local_timer_interrupts
    673.00 ± 10%     -33.7%     446.50 ±  4%  interrupts.CPU76.CAL:Function_call_interrupts
     17538           -22.4%      13612        interrupts.CPU76.LOC:Local_timer_interrupts
    681.75 ± 12%     -32.8%     458.25 ±  2%  interrupts.CPU77.CAL:Function_call_interrupts
     17543           -22.3%      13635        interrupts.CPU77.LOC:Local_timer_interrupts
    659.25 ±  2%     -28.5%     471.50 ±  9%  interrupts.CPU78.CAL:Function_call_interrupts
     17524           -22.2%      13627        interrupts.CPU78.LOC:Local_timer_interrupts
    661.25 ±  6%     -31.1%     455.75 ±  6%  interrupts.CPU79.CAL:Function_call_interrupts
     17530           -22.3%      13616        interrupts.CPU79.LOC:Local_timer_interrupts
      1150 ±  9%     -30.8%     796.00 ± 20%  interrupts.CPU8.CAL:Function_call_interrupts
     17524           -22.2%      13627        interrupts.CPU8.LOC:Local_timer_interrupts
    117.75 ± 25%     -66.0%      40.00 ± 23%  interrupts.CPU8.RES:Rescheduling_interrupts
    658.00 ± 13%     -30.3%     458.50 ±  6%  interrupts.CPU80.CAL:Function_call_interrupts
     17530           -22.3%      13629        interrupts.CPU80.LOC:Local_timer_interrupts
    654.25 ± 11%     -27.5%     474.50 ±  6%  interrupts.CPU81.CAL:Function_call_interrupts
     17543           -22.3%      13633        interrupts.CPU81.LOC:Local_timer_interrupts
    619.75 ± 10%     -26.5%     455.25 ±  5%  interrupts.CPU82.CAL:Function_call_interrupts
     17526           -22.2%      13631        interrupts.CPU82.LOC:Local_timer_interrupts
    652.00 ± 10%     -30.4%     454.00 ±  4%  interrupts.CPU83.CAL:Function_call_interrupts
     17490           -22.1%      13623        interrupts.CPU83.LOC:Local_timer_interrupts
    626.50 ± 13%     -29.1%     444.50        interrupts.CPU84.CAL:Function_call_interrupts
     17552           -22.4%      13618        interrupts.CPU84.LOC:Local_timer_interrupts
    664.00 ±  7%     -32.0%     451.75 ±  2%  interrupts.CPU85.CAL:Function_call_interrupts
     17461           -22.0%      13626        interrupts.CPU85.LOC:Local_timer_interrupts
    601.25 ±  9%     -23.2%     462.00 ±  4%  interrupts.CPU86.CAL:Function_call_interrupts
     17451           -21.9%      13633        interrupts.CPU86.LOC:Local_timer_interrupts
     17476           -22.0%      13630        interrupts.CPU87.LOC:Local_timer_interrupts
    574.00 ± 12%     -21.2%     452.25 ±  2%  interrupts.CPU88.CAL:Function_call_interrupts
     17552           -22.4%      13629        interrupts.CPU88.LOC:Local_timer_interrupts
    652.50 ±  2%     -29.6%     459.25 ±  3%  interrupts.CPU89.CAL:Function_call_interrupts
     17521           -22.2%      13626        interrupts.CPU89.LOC:Local_timer_interrupts
      1142 ±  9%     -34.3%     750.00        interrupts.CPU9.CAL:Function_call_interrupts
     17435           -22.4%      13534        interrupts.CPU9.LOC:Local_timer_interrupts
    118.50 ±  7%     -65.4%      41.00 ± 14%  interrupts.CPU9.RES:Rescheduling_interrupts
    631.75 ±  6%     -28.5%     451.50 ±  4%  interrupts.CPU90.CAL:Function_call_interrupts
     17497           -22.3%      13589        interrupts.CPU90.LOC:Local_timer_interrupts
    638.50 ± 10%     -27.7%     461.50 ±  8%  interrupts.CPU91.CAL:Function_call_interrupts
     17474           -23.1%      13437        interrupts.CPU91.LOC:Local_timer_interrupts
    653.75 ±  5%     -38.2%     404.00 ± 22%  interrupts.CPU92.CAL:Function_call_interrupts
     17514           -22.3%      13617        interrupts.CPU92.LOC:Local_timer_interrupts
    864.75 ± 36%     -47.8%     451.00 ±  8%  interrupts.CPU93.CAL:Function_call_interrupts
     17521           -22.2%      13628        interrupts.CPU93.LOC:Local_timer_interrupts
    646.25 ± 16%     -31.7%     441.25 ±  4%  interrupts.CPU94.CAL:Function_call_interrupts
     17530           -22.7%      13546        interrupts.CPU94.LOC:Local_timer_interrupts
      1023 ± 60%     -57.1%     439.00 ±  3%  interrupts.CPU95.CAL:Function_call_interrupts
     17603           -22.5%      13644        interrupts.CPU95.LOC:Local_timer_interrupts
   1681056           -22.3%    1305760        interrupts.LOC:Local_timer_interrupts
      8062 ±  6%     -63.4%       2947 ±  4%  interrupts.RES:Rescheduling_interrupts





Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


Thanks,
Oliver Sang


View attachment "config-5.10.0-rc2-00022-gfac2f60d5fe8" of type "text/plain" (171269 bytes)

View attachment "job-script" of type "text/plain" (8359 bytes)

View attachment "job.yaml" of type "text/plain" (5844 bytes)

View attachment "reproduce" of type "text/plain" (1047 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ