lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite for Android: free password hash cracker in your pocket
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <aea680a8-ad1e-f348-6555-7db1cb9c6853@linux.intel.com>
Date:   Tue, 13 Oct 2020 14:30:30 +0800
From:   Xing Zhengjun <zhengjun.xing@...ux.intel.com>
To:     kernel test robot <rong.a.chen@...el.com>,
        Josef Bacik <josef@...icpanda.com>
Cc:     David Sterba <dsterba@...e.com>,
        LKML <linux-kernel@...r.kernel.org>, lkp@...ts.01.org
Subject: Re: [LKP] Re: [btrfs] c75e839414: aim7.jobs-per-min -9.1% regression

Hi Josef,

    I re-test in v5.9, the regression still existed. Do you have time to 
take a look at this? Thanks.

On 6/15/2020 11:21 AM, Xing Zhengjun wrote:
> Hi Josef,
> 
>     Do you have time to take a look at this? Thanks.
> 
> On 6/12/2020 2:11 PM, kernel test robot wrote:
>> Greeting,
>>
>> FYI, we noticed a -9.1% regression of aim7.jobs-per-min due to commit:
>>
>>
>> commit: c75e839414d3610e6487ae3145199c500d55f7f7 ("btrfs: kill the 
>> subvol_srcu")
>> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
>>
>> in testcase: aim7
>> on test machine: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 
>> 2.30GHz with 192G memory
>> with following parameters:
>>
>>     disk: 4BRD_12G
>>     md: RAID0
>>     fs: btrfs
>>     test: disk_wrt
>>     load: 1500
>>     cpufreq_governor: performance
>>     ucode: 0x500002c
>>
>> test-description: AIM7 is a traditional UNIX system level benchmark 
>> suite which is used to test and measure the performance of multiuser 
>> system.
>> test-url: https://sourceforge.net/projects/aimbench/files/aim-suite7/
>>
>>
>>
>> If you fix the issue, kindly add following tag
>> Reported-by: kernel test robot <rong.a.chen@...el.com>
>>
>>
>> Details are as below:
>> --------------------------------------------------------------------------------------------------> 
>>
>>
>>
>> To reproduce:
>>
>>          git clone https://github.com/intel/lkp-tests.git
>>          cd lkp-tests
>>          bin/lkp install job.yaml  # job file is attached in this email
>>          bin/lkp run     job.yaml
>>
>> ========================================================================================= 
>>
>> compiler/cpufreq_governor/disk/fs/kconfig/load/md/rootfs/tbox_group/test/testcase/ucode: 
>>
>>    
>> gcc-7/performance/4BRD_12G/btrfs/x86_64-rhel-7.6/1500/RAID0/debian-x86_64-20191114.cgz/lkp-csl-2ap2/disk_wrt/aim7/0x500002c 
>>
>>
>> commit:
>>    efc3453494 ("btrfs: make btrfs_cleanup_fs_roots use the radix tree 
>> lock")
>>    c75e839414 ("btrfs: kill the subvol_srcu")
>>
>> efc3453494af7818 c75e839414d3610e6487ae31451
>> ---------------- ---------------------------
>>         fail:runs  %reproduction    fail:runs
>>             |             |             |
>>            3:9          -33%            :8     
>> dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x
>>           %stddev     %change         %stddev
>>               \          |                \
>>       29509 ±  2%      -9.1%      26837 ±  2%  aim7.jobs-per-min
>>      305.28 ±  2%     +10.0%     335.72 ±  2%  aim7.time.elapsed_time
>>      305.28 ±  2%     +10.0%     335.72 ±  2%  aim7.time.elapsed_time.max
>>     4883135 ± 10%     +37.9%    6735464 ±  7%  
>> aim7.time.involuntary_context_switches
>>       56288 ±  2%     +10.5%      62202 ±  2%  aim7.time.system_time
>>     2344783            +6.5%    2497364 ±  2%  
>> aim7.time.voluntary_context_switches
>>    62337721 ±  2%      +9.8%   68456490 ±  2%  turbostat.IRQ
>>      431.56 ±  6%     +22.3%     527.88 ±  4%  vmstat.procs.r
>>       27340 ±  2%     +11.2%      30397 ±  2%  vmstat.system.cs
>>      226804 ±  6%     +21.7%     276057 ±  4%  meminfo.Active(file)
>>      221309 ±  6%     +22.3%     270668 ±  4%  meminfo.Dirty
>>      720.89 ±111%     +49.3%       1076 ± 73%  meminfo.Mlocked
>>       14278 ±  2%      -8.3%      13094 ±  2%  meminfo.max_used_kB
>>       57228 ±  6%     +22.7%      70195 ±  5%  
>> numa-meminfo.node0.Active(file)
>>       55433 ±  6%     +21.6%      67431 ±  4%  numa-meminfo.node0.Dirty
>>       56152 ±  6%     +21.4%      68180 ±  5%  
>> numa-meminfo.node1.Active(file)
>>       55001 ±  6%     +22.5%      67397 ±  4%  numa-meminfo.node1.Dirty
>>       56373 ±  6%     +21.7%      68594 ±  4%  
>> numa-meminfo.node2.Active(file)
>>       55222 ±  7%     +22.6%      67726 ±  4%  numa-meminfo.node2.Dirty
>>       56671 ±  6%     +20.5%      68317 ±  3%  
>> numa-meminfo.node3.Active(file)
>>       55285 ±  6%     +21.8%      67355 ±  4%  numa-meminfo.node3.Dirty
>>       56694 ±  6%     +21.7%      69019 ±  4%  proc-vmstat.nr_active_file
>>       55342 ±  6%     +22.3%      67662 ±  4%  proc-vmstat.nr_dirty
>>      402316            +2.1%     410951        proc-vmstat.nr_file_pages
>>      180.22 ±111%     +49.4%     269.25 ± 73%  proc-vmstat.nr_mlock
>>       56694 ±  6%     +21.7%      69019 ±  4%  
>> proc-vmstat.nr_zone_active_file
>>       54680 ±  6%     +22.8%      67168 ±  4%  
>> proc-vmstat.nr_zone_write_pending
>>     3144381 ±  2%      +6.1%    3335275        proc-vmstat.pgactivate
>>     1387558 ±  2%      +7.9%    1496754 ±  2%  proc-vmstat.pgfault
>>      983.33 ±  4%      +5.4%       1036        
>> proc-vmstat.unevictable_pgs_culled
>>       14331 ±  6%     +22.6%      17566 ±  5%  
>> numa-vmstat.node0.nr_active_file
>>       13884 ±  6%     +21.6%      16884 ±  4%  numa-vmstat.node0.nr_dirty
>>       14330 ±  6%     +22.6%      17566 ±  5%  
>> numa-vmstat.node0.nr_zone_active_file
>>       13714 ±  6%     +22.2%      16755 ±  4%  
>> numa-vmstat.node0.nr_zone_write_pending
>>       14047 ±  6%     +21.3%      17043 ±  4%  
>> numa-vmstat.node1.nr_active_file
>>       13763 ±  6%     +22.3%      16838 ±  4%  numa-vmstat.node1.nr_dirty
>>       14047 ±  6%     +21.3%      17043 ±  4%  
>> numa-vmstat.node1.nr_zone_active_file
>>       13599 ±  6%     +23.0%      16726 ±  4%  
>> numa-vmstat.node1.nr_zone_write_pending
>>       14074 ±  5%     +21.7%      17130 ±  4%  
>> numa-vmstat.node2.nr_active_file
>>       13786 ±  6%     +22.6%      16906 ±  4%  numa-vmstat.node2.nr_dirty
>>       14074 ±  5%     +21.7%      17130 ±  4%  
>> numa-vmstat.node2.nr_zone_active_file
>>       13617 ±  7%     +23.3%      16789 ±  4%  
>> numa-vmstat.node2.nr_zone_write_pending
>>       14151 ±  6%     +20.7%      17074 ±  3%  
>> numa-vmstat.node3.nr_active_file
>>       13790 ±  6%     +22.1%      16832 ±  4%  numa-vmstat.node3.nr_dirty
>>       14152 ±  6%     +20.7%      17074 ±  3%  
>> numa-vmstat.node3.nr_zone_active_file
>>       13627 ±  6%     +22.5%      16696 ±  4%  
>> numa-vmstat.node3.nr_zone_write_pending
>>      828631 ± 16%     +49.9%    1241934 ±  8%  
>> sched_debug.cfs_rq:/.min_vruntime.stddev
>>        1.33 ± 30%     +96.9%       2.61 ± 14%  
>> sched_debug.cfs_rq:/.nr_spread_over.avg
>>      831152 ± 16%     +49.7%    1244586 ±  8%  
>> sched_debug.cfs_rq:/.spread0.stddev
>>        1.92 ± 14%     +28.4%       2.47 ±  8%  
>> sched_debug.cpu.nr_running.avg
>>        5.00 ±  9%     +29.7%       6.48 ±  9%  
>> sched_debug.cpu.nr_running.max
>>        0.90 ±  9%     +16.0%       1.05 ±  3%  
>> sched_debug.cpu.nr_running.stddev
>>       21790 ± 11%     +16.7%      25440 ±  2%  
>> sched_debug.cpu.nr_switches.avg
>>       19761 ± 11%     +17.9%      23289 ±  2%  
>> sched_debug.cpu.nr_switches.min
>>       20403 ± 12%     +18.1%      24101 ±  2%  
>> sched_debug.cpu.sched_count.avg
>>       27825 ± 10%     +15.8%      32227 ±  4%  
>> sched_debug.cpu.sched_count.max
>>       18995 ± 13%     +18.3%      22463 ±  2%  
>> sched_debug.cpu.sched_count.min
>>        1143 ± 11%     +15.6%       1322 ±  4%  
>> sched_debug.cpu.sched_count.stddev
>>        1404 ±  7%     -31.0%     969.66 ± 11%  
>> sched_debug.cpu.sched_goidle.avg
>>        1186 ±  8%     -36.0%     759.25 ± 14%  
>> sched_debug.cpu.sched_goidle.min
>>       16821 ± 14%     +20.6%      20284 ±  6%  
>> sched_debug.cpu.ttwu_count.max
>>        2790 ± 11%     +21.7%       3395 ±  7%  
>> sched_debug.cpu.ttwu_count.stddev
>>      398.19 ± 16%     +28.3%     510.73 ± 10%  
>> sched_debug.cpu.ttwu_local.stddev
>>    43238516 ±  5%     +19.0%   51459028 ±  4%  perf-stat.i.cache-misses
>>    83870573 ±  4%     +13.8%   95436555 ±  4%  
>> perf-stat.i.cache-references
>>       27181 ±  2%     +11.1%      30198 ±  2%  
>> perf-stat.i.context-switches
>>        3008 ±  2%      -7.1%       2794        perf-stat.i.cpu-migrations
>>       12612 ±  5%     -15.6%      10641 ±  3%  
>> perf-stat.i.cycles-between-cache-misses
>>     1.1e+09 ±  2%      -7.3%  1.019e+09 ±  2%  perf-stat.i.dTLB-stores
>>    10684629 ±  2%     +10.8%   11842781 ±  2%  
>> perf-stat.i.node-load-misses
>>      641429 ±  2%     +24.4%     798190 ±  2%  perf-stat.i.node-loads
>>     5915955 ±  2%      -6.3%    5543368 ±  2%  
>> perf-stat.i.node-store-misses
>>      380971            -6.7%     355629 ±  2%  perf-stat.i.node-stores
>>        1.66 ±  4%     +14.4%       1.90 ±  3%  perf-stat.overall.MPKI
>>       13064 ±  5%     -15.8%      10999 ±  4%  
>> perf-stat.overall.cycles-between-cache-misses
>>    43484558 ±  5%     +19.2%   51837476 ±  4%  perf-stat.ps.cache-misses
>>    83278267 ±  4%     +14.4%   95253129 ±  3%  
>> perf-stat.ps.cache-references
>>       27414 ±  2%     +11.2%      30486 ±  2%  
>> perf-stat.ps.context-switches
>>        3028 ±  2%      -7.0%       2817        
>> perf-stat.ps.cpu-migrations
>>   1.104e+09 ±  2%      -7.3%  1.023e+09 ±  2%  perf-stat.ps.dTLB-stores
>>        4394            -1.8%       4315        perf-stat.ps.minor-faults
>>    10766873 ±  2%     +11.0%   11947268 ±  2%  
>> perf-stat.ps.node-load-misses
>>      645940 ±  2%     +24.7%     805392 ±  2%  perf-stat.ps.node-loads
>>     5965618 ±  2%      -6.2%    5595562 ±  2%  
>> perf-stat.ps.node-store-misses
>>      383092            -6.5%     358124 ±  2%  perf-stat.ps.node-stores
>>        4395            -1.8%       4315        perf-stat.ps.page-faults
>>   1.538e+13 ±  2%     +10.0%  1.692e+13 ±  2%  
>> perf-stat.total.instructions
>>       27.39            -0.1       27.28        
>> perf-profile.calltrace.cycles-pp.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 
>>
>>       35.13            +0.1       35.23        
>> perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write 
>>
>>       35.23            +0.1       35.34        
>> perf-profile.calltrace.cycles-pp._raw_spin_lock.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter 
>>
>>       35.48            +0.1       35.60        
>> perf-profile.calltrace.cycles-pp.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write 
>>
>>       35.08            +0.1       35.22        
>> perf-profile.calltrace.cycles-pp._raw_spin_lock.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter 
>>
>>       34.97            +0.1       35.12        
>> perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write 
>>
>>       35.57            +0.1       35.72        
>> perf-profile.calltrace.cycles-pp.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 
>>
>>       35.10            +0.2       35.26        
>> perf-profile.calltrace.cycles-pp.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write 
>>
>>       35.11            +0.2       35.27        
>> perf-profile.calltrace.cycles-pp.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 
>>
>>       98.71            +0.2       98.88        
>> perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 
>>
>>       98.69            +0.2       98.87        
>> perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 
>>
>>       98.74            +0.2       98.91        
>> perf-profile.calltrace.cycles-pp.write
>>       98.71            +0.2       98.89        
>> perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write
>>       98.68            +0.2       98.86        
>> perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 
>>
>>       98.64            +0.2       98.82        
>> perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 
>>
>>       98.63            +0.2       98.81        
>> perf-profile.calltrace.cycles-pp.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64 
>>
>>       98.58            +0.2       98.77        
>> perf-profile.calltrace.cycles-pp.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write 
>>
>>       27.39            -0.1       27.28        
>> perf-profile.children.cycles-pp.btrfs_dirty_pages
>>        0.56 ±  5%      -0.1        0.46 ±  9%  
>> perf-profile.children.cycles-pp.btrfs_get_extent
>>        0.51 ±  5%      -0.1        0.41 ±  9%  
>> perf-profile.children.cycles-pp.btrfs_lookup_file_extent
>>        0.52 ±  5%      -0.1        0.43 ±  9%  
>> perf-profile.children.cycles-pp.btrfs_search_slot
>>        0.44 ±  6%      -0.1        0.35 ± 11%  
>> perf-profile.children.cycles-pp.btrfs_read_lock_root_node
>>        0.43 ±  6%      -0.1        0.34 ± 11%  
>> perf-profile.children.cycles-pp.btrfs_tree_read_lock
>>        0.19 ± 15%      -0.1        0.11 ± 27%  
>> perf-profile.children.cycles-pp.start_secondary
>>        0.19 ± 16%      -0.1        0.10 ± 27%  
>> perf-profile.children.cycles-pp.cpuidle_enter
>>        0.19 ± 16%      -0.1        0.10 ± 27%  
>> perf-profile.children.cycles-pp.cpuidle_enter_state
>>        0.41 ±  7%      -0.1        0.33 ±  8%  
>> perf-profile.children.cycles-pp.rwsem_optimistic_spin
>>        0.46 ±  7%      -0.1        0.38 ±  8%  
>> perf-profile.children.cycles-pp.rwsem_down_write_slowpath
>>        0.19 ± 15%      -0.1        0.11 ± 25%  
>> perf-profile.children.cycles-pp.secondary_startup_64
>>        0.19 ± 15%      -0.1        0.11 ± 25%  
>> perf-profile.children.cycles-pp.cpu_startup_entry
>>        0.19 ± 15%      -0.1        0.11 ± 25%  
>> perf-profile.children.cycles-pp.do_idle
>>        0.18 ± 15%      -0.1        0.10 ± 24%  
>> perf-profile.children.cycles-pp.intel_idle
>>        0.29 ±  9%      -0.1        0.22 ± 11%  
>> perf-profile.children.cycles-pp.osq_lock
>>        0.19 ±  6%      -0.0        0.15 ± 16%  
>> perf-profile.children.cycles-pp.finish_wait
>>        0.39 ±  5%      -0.0        0.35 ±  6%  
>> perf-profile.children.cycles-pp.do_unlinkat
>>        0.39 ±  6%      -0.0        0.35 ±  7%  
>> perf-profile.children.cycles-pp.unlink
>>        0.26 ±  5%      -0.0        0.22 ±  6%  
>> perf-profile.children.cycles-pp.do_sys_open
>>        0.26 ±  5%      -0.0        0.22 ±  6%  
>> perf-profile.children.cycles-pp.do_sys_openat2
>>        0.26 ±  4%      -0.0        0.22 ±  6%  
>> perf-profile.children.cycles-pp.do_filp_open
>>        0.26 ±  4%      -0.0        0.22 ±  6%  
>> perf-profile.children.cycles-pp.path_openat
>>        0.26 ±  4%      -0.0        0.22 ±  6%  
>> perf-profile.children.cycles-pp.creat
>>        0.00            +0.0        0.05        
>> perf-profile.children.cycles-pp.btrfs_calculate_inode_block_rsv_size
>>       99.65            +0.1       99.73        
>> perf-profile.children.cycles-pp.do_syscall_64
>>       99.66            +0.1       99.74        
>> perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
>>       35.62            +0.1       35.74        
>> perf-profile.children.cycles-pp.btrfs_reserve_metadata_bytes
>>       35.57            +0.1       35.72        
>> perf-profile.children.cycles-pp.btrfs_delalloc_reserve_metadata
>>       97.55            +0.2       97.72        
>> perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
>>       98.74            +0.2       98.92        
>> perf-profile.children.cycles-pp.write
>>       98.70            +0.2       98.88        
>> perf-profile.children.cycles-pp.ksys_write
>>       98.69            +0.2       98.87        
>> perf-profile.children.cycles-pp.vfs_write
>>       98.65            +0.2       98.83        
>> perf-profile.children.cycles-pp.new_sync_write
>>       98.63            +0.2       98.81        
>> perf-profile.children.cycles-pp.btrfs_file_write_iter
>>       98.59            +0.2       98.78        
>> perf-profile.children.cycles-pp.btrfs_buffered_write
>>       97.60            +0.2       97.79        
>> perf-profile.children.cycles-pp._raw_spin_lock
>>        0.18 ± 15%      -0.1        0.10 ± 24%  
>> perf-profile.self.cycles-pp.intel_idle
>>        0.28 ±  9%      -0.1        0.21 ± 12%  
>> perf-profile.self.cycles-pp.osq_lock
>>        0.45 ±  2%      -0.0        0.41 ±  2%  
>> perf-profile.self.cycles-pp._raw_spin_lock
>>      135788 ±  2%     +13.1%     153639 ±  3%  softirqs.CPU0.RCU
>>      115547 ±  3%      +8.0%     124768 ±  4%  softirqs.CPU0.TIMER
>>      135243 ±  3%     +14.0%     154188 ±  2%  softirqs.CPU1.RCU
>>      136056 ±  3%     +12.4%     152960 ±  3%  softirqs.CPU10.RCU
>>      134253 ±  3%     +14.2%     153260 ±  3%  softirqs.CPU100.RCU
>>      134668 ±  4%     +13.7%     153157 ±  2%  softirqs.CPU101.RCU
>>      109995 ±  3%      +7.9%     118637 ±  3%  softirqs.CPU101.TIMER
>>      134989 ±  4%     +13.9%     153700 ±  2%  softirqs.CPU102.RCU
>>      109876 ±  3%      +8.9%     119608 ±  2%  softirqs.CPU102.TIMER
>>      134868 ±  4%     +13.8%     153427 ±  3%  softirqs.CPU103.RCU
>>      109933 ±  3%      +8.2%     118946 ±  3%  softirqs.CPU103.TIMER
>>      134454 ±  3%     +14.0%     153285 ±  3%  softirqs.CPU104.RCU
>>      109731 ±  2%      +8.4%     118965 ±  3%  softirqs.CPU104.TIMER
>>      134878 ±  3%     +13.6%     153283 ±  3%  softirqs.CPU105.RCU
>>      134096 ±  3%     +14.1%     153022 ±  3%  softirqs.CPU106.RCU
>>      109821 ±  2%      +8.2%     118845 ±  3%  softirqs.CPU106.TIMER
>>      133905 ±  3%     +15.0%     154004 ±  3%  softirqs.CPU107.RCU
>>      134117 ±  3%     +14.3%     153232 ±  3%  softirqs.CPU108.RCU
>>      109545 ±  2%      +8.8%     119182 ±  3%  softirqs.CPU108.TIMER
>>      134574 ±  3%     +13.8%     153202 ±  3%  softirqs.CPU109.RCU
>>      109804 ±  3%      +8.2%     118760 ±  3%  softirqs.CPU109.TIMER
>>      135106 ±  3%     +14.8%     155124 ±  3%  softirqs.CPU11.RCU
>>      111746 ±  4%      +7.8%     120460 ±  3%  softirqs.CPU11.TIMER
>>      134482 ±  4%     +14.3%     153687 ±  3%  softirqs.CPU110.RCU
>>      110314 ±  3%      +8.1%     119241 ±  3%  softirqs.CPU110.TIMER
>>      134464 ±  3%     +14.3%     153634 ±  3%  softirqs.CPU111.RCU
>>      110626 ±  3%      +8.3%     119815 ±  3%  softirqs.CPU111.TIMER
>>      134895 ±  3%     +14.4%     154257 ±  3%  softirqs.CPU112.RCU
>>      110040 ±  3%     +14.6%     126058 ± 16%  softirqs.CPU112.TIMER
>>      134644 ±  3%     +13.6%     152993 ±  3%  softirqs.CPU113.RCU
>>      109693 ±  3%      +8.1%     118569 ±  3%  softirqs.CPU113.TIMER
>>      134313 ±  3%     +13.9%     152993 ±  3%  softirqs.CPU114.RCU
>>      109711 ±  3%      +8.6%     119143 ±  3%  softirqs.CPU114.TIMER
>>      134559 ±  3%     +13.8%     153115 ±  3%  softirqs.CPU115.RCU
>>      110226 ±  3%      +8.2%     119221 ±  3%  softirqs.CPU115.TIMER
>>      134087 ±  3%     +14.2%     153077 ±  3%  softirqs.CPU116.RCU
>>      110225 ±  3%      +9.0%     120184 ±  3%  softirqs.CPU116.TIMER
>>      134271 ±  3%     +14.1%     153220 ±  3%  softirqs.CPU117.RCU
>>      109179 ±  2%      +8.3%     118274 ±  3%  softirqs.CPU117.TIMER
>>      134281 ±  3%     +14.1%     153230 ±  3%  softirqs.CPU118.RCU
>>      134415 ±  3%     +14.0%     153215 ±  3%  softirqs.CPU119.RCU
>>      109514 ±  2%      +8.3%     118643 ±  3%  softirqs.CPU119.TIMER
>>      135560 ±  3%     +13.5%     153874 ±  3%  softirqs.CPU12.RCU
>>      110316 ±  2%      +8.5%     119734 ±  3%  softirqs.CPU12.TIMER
>>      134177 ±  3%     +14.6%     153815 ±  3%  softirqs.CPU120.RCU
>>      134367 ±  3%     +15.0%     154493 ±  3%  softirqs.CPU121.RCU
>>      133972 ±  3%     +14.3%     153173 ±  3%  softirqs.CPU122.RCU
>>      134801 ±  4%     +13.6%     153136 ±  3%  softirqs.CPU123.RCU
>>      134709 ±  3%     +13.9%     153440 ±  3%  softirqs.CPU124.RCU
>>      135185 ±  3%     +13.1%     152925 ±  3%  softirqs.CPU125.RCU
>>      134977 ±  3%     +13.8%     153608 ±  3%  softirqs.CPU126.RCU
>>      133862 ±  4%     +14.7%     153558 ±  3%  softirqs.CPU127.RCU
>>      134554 ±  3%     +13.9%     153237 ±  3%  softirqs.CPU128.RCU
>>      134254 ±  3%     +14.1%     153130 ±  3%  softirqs.CPU129.RCU
>>      135133 ±  3%     +14.0%     154103 ±  3%  softirqs.CPU13.RCU
>>      110389 ±  2%      +8.1%     119382 ±  3%  softirqs.CPU13.TIMER
>>      134319 ±  3%     +13.8%     152909 ±  3%  softirqs.CPU130.RCU
>>      133590 ±  3%     +14.5%     152925 ±  3%  softirqs.CPU131.RCU
>>      134416 ±  4%     +14.0%     153223 ±  3%  softirqs.CPU132.RCU
>>      133931 ±  3%     +14.5%     153359 ±  3%  softirqs.CPU133.RCU
>>      134031 ±  3%     +14.5%     153527 ±  3%  softirqs.CPU134.RCU
>>      133851 ±  4%     +14.3%     153052 ±  3%  softirqs.CPU135.RCU
>>      134235 ±  4%     +13.9%     152959 ±  3%  softirqs.CPU136.RCU
>>      134003 ±  3%     +14.2%     153037 ±  3%  softirqs.CPU137.RCU
>>      133453 ±  3%     +14.4%     152625 ±  3%  softirqs.CPU138.RCU
>>      133514 ±  3%     +14.8%     153289 ±  3%  softirqs.CPU139.RCU
>>      134952 ±  3%     +13.6%     153372 ±  3%  softirqs.CPU14.RCU
>>      134045 ±  3%     +13.7%     152361 ±  3%  softirqs.CPU140.RCU
>>      133662 ±  3%     +14.1%     152528 ±  3%  softirqs.CPU141.RCU
>>      133643 ±  3%     +14.7%     153283 ±  2%  softirqs.CPU142.RCU
>>      133313 ±  3%     +15.0%     153254 ±  3%  softirqs.CPU143.RCU
>>      133585 ±  3%     +14.4%     152837 ±  3%  softirqs.CPU144.RCU
>>      108590 ±  3%     +12.7%     122339 ±  5%  softirqs.CPU144.TIMER
>>      133471 ±  3%     +14.5%     152883 ±  3%  softirqs.CPU145.RCU
>>      108095 ±  3%     +14.5%     123726 ±  6%  softirqs.CPU145.TIMER
>>      133588 ±  3%     +15.4%     154173 ±  3%  softirqs.CPU146.RCU
>>      108268 ±  3%     +19.8%     129684 ± 16%  softirqs.CPU146.TIMER
>>      134060 ±  3%     +14.3%     153270 ±  3%  softirqs.CPU147.RCU
>>      108731 ±  3%     +13.0%     122902 ±  5%  softirqs.CPU147.TIMER
>>      133778 ±  3%     +14.1%     152681 ±  3%  softirqs.CPU148.RCU
>>      108208 ±  3%     +12.6%     121875 ±  5%  softirqs.CPU148.TIMER
>>      133514 ±  3%     +14.2%     152460 ±  3%  softirqs.CPU149.RCU
>>      108054 ±  3%     +15.6%     124938 ±  9%  softirqs.CPU149.TIMER
>>      134677 ±  3%     +14.5%     154138 ±  2%  softirqs.CPU15.RCU
>>      133245 ±  3%     +14.4%     152373 ±  3%  softirqs.CPU150.RCU
>>      107788 ±  3%     +13.1%     121957 ±  5%  softirqs.CPU150.TIMER
>>      133747 ±  3%     +14.0%     152473 ±  3%  softirqs.CPU151.RCU
>>      108258 ±  4%     +12.4%     121662 ±  5%  softirqs.CPU151.TIMER
>>      133819 ±  3%     +14.1%     152726 ±  3%  softirqs.CPU152.RCU
>>      108163 ±  3%     +12.7%     121940 ±  5%  softirqs.CPU152.TIMER
>>      134057 ±  3%     +14.0%     152844 ±  3%  softirqs.CPU153.RCU
>>      107680 ±  3%     +13.1%     121787 ±  5%  softirqs.CPU153.TIMER
>>      133660 ±  3%     +14.2%     152675 ±  3%  softirqs.CPU154.RCU
>>      107911 ±  3%     +15.4%     124485 ±  9%  softirqs.CPU154.TIMER
>>      133349 ±  3%     +14.6%     152811 ±  3%  softirqs.CPU155.RCU
>>      107513 ±  3%     +15.1%     123737 ±  6%  softirqs.CPU155.TIMER
>>      133901 ±  3%     +14.0%     152615 ±  3%  softirqs.CPU156.RCU
>>      107853 ±  3%     +13.0%     121904 ±  5%  softirqs.CPU156.TIMER
>>      133621 ±  3%     +14.2%     152615 ±  3%  softirqs.CPU157.RCU
>>      108040 ±  3%     +12.6%     121684 ±  5%  softirqs.CPU157.TIMER
>>      133891 ±  3%     +14.2%     152862 ±  3%  softirqs.CPU158.RCU
>>      108534 ±  3%     +12.7%     122300 ±  5%  softirqs.CPU158.TIMER
>>      133925 ±  3%     +14.5%     153347 ±  3%  softirqs.CPU159.RCU
>>      108099 ±  3%     +12.9%     122015 ±  5%  softirqs.CPU159.TIMER
>>      134893 ±  3%     +14.5%     154389 ±  3%  softirqs.CPU16.RCU
>>      134169 ±  3%     +13.9%     152812 ±  3%  softirqs.CPU160.RCU
>>      108300 ±  3%     +12.9%     122318 ±  5%  softirqs.CPU160.TIMER
>>      133818 ±  3%     +15.1%     154048 ±  4%  softirqs.CPU161.RCU
>>      107919 ±  3%     +13.6%     122606 ±  6%  softirqs.CPU161.TIMER
>>      133867 ±  3%     +14.2%     152856 ±  3%  softirqs.CPU162.RCU
>>      107991 ±  3%     +13.2%     122209 ±  5%  softirqs.CPU162.TIMER
>>      133579 ±  3%     +14.3%     152654 ±  3%  softirqs.CPU163.RCU
>>      108317 ±  3%     +12.9%     122271 ±  5%  softirqs.CPU163.TIMER
>>      133914 ±  3%     +14.4%     153158 ±  3%  softirqs.CPU164.RCU
>>      133684 ±  3%     +14.4%     152895 ±  3%  softirqs.CPU165.RCU
>>      107435 ±  3%     +12.5%     120877 ±  5%  softirqs.CPU165.TIMER
>>      133983 ±  3%     +13.9%     152582 ±  3%  softirqs.CPU166.RCU
>>      107432 ±  3%     +13.2%     121641 ±  5%  softirqs.CPU166.TIMER
>>      133947 ±  3%     +13.6%     152137 ±  3%  softirqs.CPU167.RCU
>>      134005 ±  3%     +14.5%     153385 ±  4%  softirqs.CPU168.RCU
>>      115752 ±  4%     +12.0%     129634 ±  4%  softirqs.CPU168.TIMER
>>      133557 ±  3%     +15.2%     153865 ±  4%  softirqs.CPU169.RCU
>>      115884 ±  3%     +11.7%     129481 ±  4%  softirqs.CPU169.TIMER
>>      135348 ±  3%     +13.8%     153977 ±  3%  softirqs.CPU17.RCU
>>      110589 ±  3%      +7.9%     119298 ±  3%  softirqs.CPU17.TIMER
>>      134463 ±  3%     +14.0%     153229 ±  3%  softirqs.CPU170.RCU
>>      133740 ±  3%     +14.1%     152630 ±  3%  softirqs.CPU171.RCU
>>      116524 ±  4%     +10.9%     129237 ±  3%  softirqs.CPU171.TIMER
>>      133716 ±  3%     +13.8%     152196 ±  3%  softirqs.CPU172.RCU
>>      116393 ±  4%     +10.9%     129087 ±  4%  softirqs.CPU172.TIMER
>>      133564 ±  3%     +13.9%     152165 ±  3%  softirqs.CPU173.RCU
>>      116237 ±  3%     +11.4%     129460 ±  3%  softirqs.CPU173.TIMER
>>      133411 ±  3%     +14.5%     152764 ±  3%  softirqs.CPU174.RCU
>>      116018 ±  3%     +11.2%     129006 ±  4%  softirqs.CPU174.TIMER
>>      133147 ±  3%     +15.0%     153104 ±  2%  softirqs.CPU175.RCU
>>      116052 ±  4%     +11.7%     129664 ±  3%  softirqs.CPU175.TIMER
>>      133989 ±  3%     +13.8%     152504 ±  3%  softirqs.CPU176.RCU
>>      116071 ±  4%     +10.9%     128756 ±  3%  softirqs.CPU176.TIMER
>>      133902 ±  3%     +14.1%     152811 ±  3%  softirqs.CPU177.RCU
>>      115994 ±  3%     +11.1%     128849 ±  4%  softirqs.CPU177.TIMER
>>      133679 ±  3%     +14.4%     152887 ±  3%  softirqs.CPU178.RCU
>>      115942 ±  4%     +13.4%     131439 ±  5%  softirqs.CPU178.TIMER
>>      133985 ±  3%     +14.0%     152762 ±  3%  softirqs.CPU179.RCU
>>      116347 ±  4%     +11.2%     129422 ±  4%  softirqs.CPU179.TIMER
>>      134982 ±  3%     +14.0%     153838 ±  3%  softirqs.CPU18.RCU
>>      110310 ±  3%      +8.6%     119851 ±  3%  softirqs.CPU18.TIMER
>>      133236 ±  3%     +14.7%     152870 ±  3%  softirqs.CPU180.RCU
>>      115901 ±  4%     +11.3%     129054 ±  4%  softirqs.CPU180.TIMER
>>      133615 ±  3%     +14.4%     152876 ±  3%  softirqs.CPU181.RCU
>>      115505 ±  4%     +11.3%     128560 ±  3%  softirqs.CPU181.TIMER
>>      133171 ±  3%     +14.7%     152810 ±  3%  softirqs.CPU182.RCU
>>      115955 ±  4%     +11.0%     128687 ±  3%  softirqs.CPU182.TIMER
>>      133413 ±  3%     +14.4%     152671 ±  3%  softirqs.CPU183.RCU
>>      116313 ±  4%     +11.0%     129140 ±  3%  softirqs.CPU183.TIMER
>>      133598 ±  3%     +14.8%     153388 ±  3%  softirqs.CPU184.RCU
>>      115462 ±  3%     +13.0%     130430 ±  6%  softirqs.CPU184.TIMER
>>      133884 ±  3%     +13.9%     152525 ±  3%  softirqs.CPU185.RCU
>>      115598 ±  4%     +11.1%     128397 ±  4%  softirqs.CPU185.TIMER
>>      133262 ±  3%     +14.5%     152646 ±  3%  softirqs.CPU186.RCU
>>      115745 ±  3%     +11.1%     128612 ±  4%  softirqs.CPU186.TIMER
>>      133430 ±  3%     +14.5%     152767 ±  3%  softirqs.CPU187.RCU
>>      115769 ±  4%     +11.0%     128464 ±  3%  softirqs.CPU187.TIMER
>>      133259 ±  3%     +14.8%     153036 ±  3%  softirqs.CPU188.RCU
>>      116108 ±  4%     +11.3%     129232 ±  3%  softirqs.CPU188.TIMER
>>      133609 ±  3%     +14.1%     152436 ±  3%  softirqs.CPU189.RCU
>>      115984 ±  4%     +11.0%     128710 ±  3%  softirqs.CPU189.TIMER
>>      135090 ±  3%     +13.9%     153802 ±  3%  softirqs.CPU19.RCU
>>      110865 ±  2%      +8.2%     119936 ±  3%  softirqs.CPU19.TIMER
>>      133685 ±  3%     +14.2%     152631 ±  3%  softirqs.CPU190.RCU
>>      115642 ±  3%     +11.2%     128581 ±  4%  softirqs.CPU190.TIMER
>>      131031 ±  3%     +14.6%     150126 ±  3%  softirqs.CPU191.RCU
>>      114275 ±  4%     +11.4%     127249 ±  4%  softirqs.CPU191.TIMER
>>      134928 ±  3%     +14.3%     154165 ±  3%  softirqs.CPU2.RCU
>>      135126 ±  3%     +14.0%     154083 ±  3%  softirqs.CPU20.RCU
>>      111013 ±  3%      +8.6%     120549 ±  3%  softirqs.CPU20.TIMER
>>      134663 ±  3%     +14.5%     154251 ±  3%  softirqs.CPU21.RCU
>>      109671 ±  2%      +8.2%     118708 ±  3%  softirqs.CPU21.TIMER
>>      135047 ±  3%     +13.9%     153766 ±  3%  softirqs.CPU22.RCU
>>      134437 ±  3%     +14.2%     153492 ±  3%  softirqs.CPU23.RCU
>>      109675 ±  2%      +8.3%     118824 ±  3%  softirqs.CPU23.TIMER
>>      136114 ±  3%     +13.3%     154277 ±  3%  softirqs.CPU24.RCU
>>      134825 ±  3%     +14.2%     154007 ±  3%  softirqs.CPU25.RCU
>>      139234 ±  4%     +13.8%     158515 ±  3%  softirqs.CPU26.RCU
>>      135056 ±  3%     +14.1%     154062 ±  3%  softirqs.CPU27.RCU
>>      135307 ±  3%     +13.7%     153897 ±  3%  softirqs.CPU28.RCU
>>      134686 ±  3%     +14.7%     154497 ±  3%  softirqs.CPU29.RCU
>>      134688 ±  3%     +14.4%     154115 ±  2%  softirqs.CPU3.RCU
>>      134642 ±  3%     +14.5%     154226 ±  3%  softirqs.CPU30.RCU
>>      134773 ±  3%     +14.2%     153887 ±  3%  softirqs.CPU31.RCU
>>      134523 ±  3%     +14.3%     153733 ±  3%  softirqs.CPU32.RCU
>>      134488 ±  3%     +14.4%     153801 ±  3%  softirqs.CPU33.RCU
>>      134483 ±  3%     +14.1%     153380 ±  3%  softirqs.CPU34.RCU
>>      134886 ±  3%     +13.6%     153180 ±  3%  softirqs.CPU35.RCU
>>      135021 ±  3%     +13.6%     153340 ±  3%  softirqs.CPU36.RCU
>>      134064 ±  3%     +15.1%     154277 ±  3%  softirqs.CPU37.RCU
>>      134166 ±  3%     +14.8%     153967 ±  3%  softirqs.CPU38.RCU
>>      134488 ±  3%     +14.4%     153871 ±  3%  softirqs.CPU39.RCU
>>      135294 ±  2%     +14.6%     155025 ±  3%  softirqs.CPU4.RCU
>>      134598 ±  4%     +14.3%     153848 ±  3%  softirqs.CPU40.RCU
>>      134255 ±  3%     +14.8%     154099 ±  3%  softirqs.CPU41.RCU
>>      134334 ±  3%     +14.3%     153491 ±  3%  softirqs.CPU42.RCU
>>      134201 ±  3%     +14.5%     153654 ±  3%  softirqs.CPU43.RCU
>>      134260 ±  3%     +14.2%     153359 ±  3%  softirqs.CPU44.RCU
>>      134005 ±  3%     +14.4%     153367 ±  3%  softirqs.CPU45.RCU
>>      134236 ±  3%     +14.3%     153419 ±  3%  softirqs.CPU46.RCU
>>      134552 ±  3%     +13.8%     153056 ±  3%  softirqs.CPU47.RCU
>>      134623 ±  3%     +14.4%     154028 ±  3%  softirqs.CPU48.RCU
>>      134348 ±  3%     +14.7%     154080 ±  3%  softirqs.CPU49.RCU
>>      109020 ±  3%     +18.6%     129279 ± 14%  softirqs.CPU49.TIMER
>>      134334 ±  3%     +15.0%     154455 ±  3%  softirqs.CPU5.RCU
>>      110260 ±  2%      +8.8%     119951 ±  4%  softirqs.CPU5.TIMER
>>      134377 ±  3%     +13.8%     152954 ±  3%  softirqs.CPU50.RCU
>>      109169 ±  3%     +15.0%     125548 ±  9%  softirqs.CPU50.TIMER
>>      134780 ±  3%     +14.2%     153967 ±  3%  softirqs.CPU51.RCU
>>      109951 ±  3%     +12.5%     123641 ±  5%  softirqs.CPU51.TIMER
>>      133929 ±  3%     +15.0%     154012 ±  3%  softirqs.CPU52.RCU
>>      108665 ±  3%     +12.8%     122613 ±  5%  softirqs.CPU52.TIMER
>>      133761 ±  3%     +14.9%     153675 ±  3%  softirqs.CPU53.RCU
>>      108606 ±  3%     +19.6%     129861 ± 16%  softirqs.CPU53.TIMER
>>      134369 ±  3%     +14.1%     153255 ±  3%  softirqs.CPU54.RCU
>>      108917 ±  3%     +12.5%     122585 ±  5%  softirqs.CPU54.TIMER
>>      133810 ±  3%     +14.6%     153282 ±  3%  softirqs.CPU55.RCU
>>      108875 ±  3%     +12.5%     122468 ±  5%  softirqs.CPU55.TIMER
>>      134038 ±  3%     +14.3%     153248 ±  3%  softirqs.CPU56.RCU
>>      108569 ±  3%     +12.8%     122502 ±  5%  softirqs.CPU56.TIMER
>>      133993 ±  4%     +14.5%     153426 ±  3%  softirqs.CPU57.RCU
>>      108465 ±  3%     +12.8%     122399 ±  5%  softirqs.CPU57.TIMER
>>      134049 ±  3%     +14.7%     153708 ±  3%  softirqs.CPU58.RCU
>>      108634 ±  4%     +18.8%     129107 ± 16%  softirqs.CPU58.TIMER
>>      134025 ±  3%     +14.9%     154015 ±  3%  softirqs.CPU59.RCU
>>      108501 ±  3%     +18.6%     128721 ± 13%  softirqs.CPU59.TIMER
>>      135247 ±  3%     +14.5%     154804 ±  3%  softirqs.CPU6.RCU
>>      133745 ±  4%     +14.7%     153452 ±  3%  softirqs.CPU60.RCU
>>      108526 ±  3%     +13.0%     122688 ±  5%  softirqs.CPU60.TIMER
>>      133931 ±  3%     +14.6%     153549 ±  3%  softirqs.CPU61.RCU
>>      108764 ±  3%     +12.9%     122787 ±  5%  softirqs.CPU61.TIMER
>>      134682 ±  3%     +13.8%     153266 ±  3%  softirqs.CPU62.RCU
>>      109188 ±  3%     +12.5%     122806 ±  5%  softirqs.CPU62.TIMER
>>      134190 ±  3%     +14.6%     153761 ±  2%  softirqs.CPU63.RCU
>>      108489 ±  3%     +12.8%     122426 ±  5%  softirqs.CPU63.TIMER
>>      134280 ±  3%     +14.3%     153538 ±  3%  softirqs.CPU64.RCU
>>      108493 ±  3%     +13.3%     122953 ±  5%  softirqs.CPU64.TIMER
>>      134074 ±  3%     +14.7%     153765 ±  3%  softirqs.CPU65.RCU
>>      108331 ±  3%     +13.0%     122440 ±  5%  softirqs.CPU65.TIMER
>>      134424 ±  4%     +14.3%     153692 ±  3%  softirqs.CPU66.RCU
>>      108874 ±  3%     +12.5%     122490 ±  5%  softirqs.CPU66.TIMER
>>      133801 ±  3%     +14.9%     153746 ±  3%  softirqs.CPU67.RCU
>>      108903 ±  3%     +13.2%     123240 ±  5%  softirqs.CPU67.TIMER
>>      134085 ±  3%     +14.4%     153371 ±  3%  softirqs.CPU68.RCU
>>      109310 ±  3%     +12.3%     122810 ±  5%  softirqs.CPU68.TIMER
>>      134091 ±  3%     +14.5%     153515 ±  3%  softirqs.CPU69.RCU
>>      108193 ±  3%     +12.5%     121716 ±  5%  softirqs.CPU69.TIMER
>>      134883 ±  3%     +13.6%     153181 ±  3%  softirqs.CPU7.RCU
>>      134392 ±  3%     +14.8%     154240 ±  3%  softirqs.CPU70.RCU
>>      108056 ±  3%     +13.4%     122578 ±  5%  softirqs.CPU70.TIMER
>>      133901 ±  3%     +14.2%     152873 ±  3%  softirqs.CPU71.RCU
>>      134273 ±  3%     +14.4%     153630 ±  3%  softirqs.CPU72.RCU
>>      116189 ±  3%     +11.5%     129495 ±  3%  softirqs.CPU72.TIMER
>>      134231 ±  3%     +14.5%     153682 ±  3%  softirqs.CPU73.RCU
>>      116348 ±  3%     +11.3%     129453 ±  3%  softirqs.CPU73.TIMER
>>      134235 ±  3%     +14.1%     153158 ±  3%  softirqs.CPU74.RCU
>>      133896 ±  3%     +14.6%     153477 ±  3%  softirqs.CPU75.RCU
>>      116933 ±  4%     +11.2%     129980 ±  3%  softirqs.CPU75.TIMER
>>      134014 ±  3%     +14.3%     153198 ±  3%  softirqs.CPU76.RCU
>>      116889 ±  3%     +11.0%     129768 ±  3%  softirqs.CPU76.TIMER
>>      134608 ±  3%     +13.7%     152994 ±  3%  softirqs.CPU77.RCU
>>      117342 ±  3%     +11.0%     130274 ±  3%  softirqs.CPU77.TIMER
>>      134551 ±  3%     +13.9%     153253 ±  3%  softirqs.CPU78.RCU
>>      117229 ±  4%     +10.5%     129549 ±  4%  softirqs.CPU78.TIMER
>>      134118 ±  3%     +14.3%     153325 ±  3%  softirqs.CPU79.RCU
>>      116982 ±  3%     +11.0%     129802 ±  3%  softirqs.CPU79.TIMER
>>      135303 ±  3%     +13.9%     154129 ±  3%  softirqs.CPU8.RCU
>>      134699 ±  3%     +13.6%     153047 ±  3%  softirqs.CPU80.RCU
>>      117058 ±  4%     +10.6%     129470 ±  3%  softirqs.CPU80.TIMER
>>      134088 ±  3%     +15.6%     155063 ±  3%  softirqs.CPU81.RCU
>>      116539 ±  4%     +12.3%     130886 ±  5%  softirqs.CPU81.TIMER
>>      133989 ±  3%     +14.6%     153526 ±  3%  softirqs.CPU82.RCU
>>      116614 ±  4%     +16.8%     136238 ± 12%  softirqs.CPU82.TIMER
>>      133966 ±  3%     +14.3%     153142 ±  3%  softirqs.CPU83.RCU
>>      117140 ±  3%     +11.2%     130205 ±  4%  softirqs.CPU83.TIMER
>>      133561 ±  3%     +15.0%     153661 ±  3%  softirqs.CPU84.RCU
>>      116648 ±  4%     +11.3%     129844 ±  4%  softirqs.CPU84.TIMER
>>      134217 ±  3%     +14.2%     153330 ±  3%  softirqs.CPU85.RCU
>>      116247 ±  4%     +11.3%     129336 ±  3%  softirqs.CPU85.TIMER
>>      133956 ±  3%     +14.5%     153338 ±  3%  softirqs.CPU86.RCU
>>      116654 ±  4%     +11.1%     129560 ±  3%  softirqs.CPU86.TIMER
>>      133834 ±  3%     +15.1%     154049 ±  2%  softirqs.CPU87.RCU
>>      116898 ±  4%     +12.0%     130912 ±  3%  softirqs.CPU87.TIMER
>>      133951 ±  3%     +15.1%     154211 ±  3%  softirqs.CPU88.RCU
>>      116139 ±  3%     +16.0%     134676 ± 13%  softirqs.CPU88.TIMER
>>      134288 ±  3%     +14.3%     153433 ±  2%  softirqs.CPU89.RCU
>>      116227 ±  3%     +11.0%     129064 ±  3%  softirqs.CPU89.TIMER
>>      135289 ±  3%     +13.6%     153744 ±  3%  softirqs.CPU9.RCU
>>      133598 ±  3%     +14.5%     153019 ±  3%  softirqs.CPU90.RCU
>>      116395 ±  3%     +10.9%     129084 ±  4%  softirqs.CPU90.TIMER
>>      133854 ±  4%     +14.9%     153799 ±  3%  softirqs.CPU91.RCU
>>      116562 ±  4%     +11.1%     129452 ±  3%  softirqs.CPU91.TIMER
>>      134107 ±  3%     +14.1%     152958 ±  3%  softirqs.CPU92.RCU
>>      117107 ±  4%     +10.6%     129559 ±  3%  softirqs.CPU92.TIMER
>>      133800 ±  3%     +14.7%     153465 ±  3%  softirqs.CPU93.RCU
>>      116415 ±  3%     +11.5%     129823 ±  4%  softirqs.CPU93.TIMER
>>      134092 ±  3%     +14.0%     152835 ±  3%  softirqs.CPU94.RCU
>>      116063 ±  3%     +11.4%     129252 ±  4%  softirqs.CPU94.TIMER
>>      134284 ±  3%     +14.1%     153230 ±  4%  softirqs.CPU95.RCU
>>      117908 ±  3%     +11.0%     130904 ±  4%  softirqs.CPU95.TIMER
>>      134264 ±  3%     +14.3%     153455 ±  3%  softirqs.CPU96.RCU
>>      109819 ±  3%      +8.6%     119258 ±  3%  softirqs.CPU96.TIMER
>>      135000 ±  3%     +13.7%     153529 ±  3%  softirqs.CPU97.RCU
>>      134163 ±  3%     +14.2%     153253 ±  3%  softirqs.CPU98.RCU
>>      109990 ±  3%      +8.1%     118896 ±  3%  softirqs.CPU98.TIMER
>>      134966 ±  3%     +13.5%     153245 ±  3%  softirqs.CPU99.RCU
>>    25776297 ±  3%     +14.3%   29452390 ±  3%  softirqs.RCU
>>    21445010 ±  3%      +9.5%   23488096 ±  3%  softirqs.TIMER
>>      614.00 ±  2%      +9.9%     674.50 ±  2%  
>> interrupts.9:IO-APIC.9-fasteoi.acpi
>>      611854 ±  2%     +10.1%     673945 ±  2%  
>> interrupts.CPU0.LOC:Local_timer_interrupts
>>      614.00 ±  2%      +9.9%     674.50 ±  2%  
>> interrupts.CPU1.9:IO-APIC.9-fasteoi.acpi
>>      611835 ±  2%     +10.1%     673883 ±  2%  
>> interrupts.CPU1.LOC:Local_timer_interrupts
>>      611859 ±  2%     +10.2%     673975 ±  2%  
>> interrupts.CPU10.LOC:Local_timer_interrupts
>>        7508 ±  5%     +15.5%       8675 ±  4%  
>> interrupts.CPU10.RES:Rescheduling_interrupts
>>      611802 ±  2%     +10.2%     673904 ±  2%  
>> interrupts.CPU100.LOC:Local_timer_interrupts
>>        7323 ±  5%     +16.1%       8503 ±  3%  
>> interrupts.CPU100.RES:Rescheduling_interrupts
>>      611804 ±  2%     +10.2%     673917 ±  2%  
>> interrupts.CPU101.LOC:Local_timer_interrupts
>>        7342 ±  3%     +13.8%       8352 ±  3%  
>> interrupts.CPU101.RES:Rescheduling_interrupts
>>      611780 ±  2%     +10.2%     673877 ±  2%  
>> interrupts.CPU102.LOC:Local_timer_interrupts
>>        7506 ±  5%     +12.2%       8424 ±  4%  
>> interrupts.CPU102.RES:Rescheduling_interrupts
>>      611734 ±  2%     +10.2%     673851 ±  2%  
>> interrupts.CPU103.LOC:Local_timer_interrupts
>>        7436 ±  3%     +15.8%       8607 ±  4%  
>> interrupts.CPU103.RES:Rescheduling_interrupts
>>      611800 ±  2%     +10.2%     673908 ±  2%  
>> interrupts.CPU104.LOC:Local_timer_interrupts
>>        7519 ±  5%     +13.1%       8502 ±  2%  
>> interrupts.CPU104.RES:Rescheduling_interrupts
>>      611790 ±  2%     +10.1%     673864 ±  2%  
>> interrupts.CPU105.LOC:Local_timer_interrupts
>>        7346 ±  5%     +16.4%       8553 ±  2%  
>> interrupts.CPU105.RES:Rescheduling_interrupts
>>      611801 ±  2%     +10.1%     673859 ±  2%  
>> interrupts.CPU106.LOC:Local_timer_interrupts
>>        7352 ±  5%     +16.5%       8567 ±  2%  
>> interrupts.CPU106.RES:Rescheduling_interrupts
>>      611797 ±  2%     +10.1%     673890 ±  2%  
>> interrupts.CPU107.LOC:Local_timer_interrupts
>>        7238 ±  6%     +15.3%       8344 ±  4%  
>> interrupts.CPU107.RES:Rescheduling_interrupts
>>      611778 ±  2%     +10.2%     673882 ±  2%  
>> interrupts.CPU108.LOC:Local_timer_interrupts
>>        7310 ±  5%     +16.3%       8503 ±  5%  
>> interrupts.CPU108.RES:Rescheduling_interrupts
>>      611788 ±  2%     +10.1%     673881 ±  2%  
>> interrupts.CPU109.LOC:Local_timer_interrupts
>>        7324 ±  6%     +13.4%       8307        
>> interrupts.CPU109.RES:Rescheduling_interrupts
>>      611827 ±  2%     +10.2%     673960 ±  2%  
>> interrupts.CPU11.LOC:Local_timer_interrupts
>>        7598 ±  6%     +12.6%       8557 ±  5%  
>> interrupts.CPU11.RES:Rescheduling_interrupts
>>      611796 ±  2%     +10.1%     673865 ±  2%  
>> interrupts.CPU110.LOC:Local_timer_interrupts
>>        7326 ±  4%     +16.2%       8513 ±  4%  
>> interrupts.CPU110.RES:Rescheduling_interrupts
>>      611804 ±  2%     +10.1%     673890 ±  2%  
>> interrupts.CPU111.LOC:Local_timer_interrupts
>>        7312 ±  3%     +14.4%       8365 ±  3%  
>> interrupts.CPU111.RES:Rescheduling_interrupts
>>      611801 ±  2%     +10.1%     673890 ±  2%  
>> interrupts.CPU112.LOC:Local_timer_interrupts
>>        7277 ±  2%     +14.6%       8337 ±  5%  
>> interrupts.CPU112.RES:Rescheduling_interrupts
>>      611789 ±  2%     +10.2%     673896 ±  2%  
>> interrupts.CPU113.LOC:Local_timer_interrupts
>>        7346 ±  4%     +15.2%       8461 ±  4%  
>> interrupts.CPU113.RES:Rescheduling_interrupts
>>      611759 ±  2%     +10.2%     673861 ±  2%  
>> interrupts.CPU114.LOC:Local_timer_interrupts
>>        7290 ±  4%     +17.9%       8594 ±  4%  
>> interrupts.CPU114.RES:Rescheduling_interrupts
>>      611783 ±  2%     +10.2%     673883 ±  2%  
>> interrupts.CPU115.LOC:Local_timer_interrupts
>>        7470 ±  4%     +13.6%       8489 ±  5%  
>> interrupts.CPU115.RES:Rescheduling_interrupts
>>      611807 ±  2%     +10.2%     673925 ±  2%  
>> interrupts.CPU116.LOC:Local_timer_interrupts
>>        7336 ±  4%     +15.3%       8455 ±  2%  
>> interrupts.CPU116.RES:Rescheduling_interrupts
>>      611778 ±  2%     +10.2%     673890 ±  2%  
>> interrupts.CPU117.LOC:Local_timer_interrupts
>>        7326 ±  5%     +15.9%       8493 ±  3%  
>> interrupts.CPU117.RES:Rescheduling_interrupts
>>      611782 ±  2%     +10.1%     673849 ±  2%  
>> interrupts.CPU118.LOC:Local_timer_interrupts
>>        7239 ±  4%     +18.4%       8574 ±  5%  
>> interrupts.CPU118.RES:Rescheduling_interrupts
>>      611778 ±  2%     +10.2%     673881 ±  2%  
>> interrupts.CPU119.LOC:Local_timer_interrupts
>>        7327 ±  4%     +13.7%       8335 ±  6%  
>> interrupts.CPU119.RES:Rescheduling_interrupts
>>      611853 ±  2%     +10.1%     673944 ±  2%  
>> interrupts.CPU12.LOC:Local_timer_interrupts
>>        7516 ±  4%     +17.0%       8798 ±  6%  
>> interrupts.CPU12.RES:Rescheduling_interrupts
>>      611417 ±  2%     +10.2%     673606 ±  2%  
>> interrupts.CPU120.LOC:Local_timer_interrupts
>>        7392 ±  4%     +14.2%       8444 ±  5%  
>> interrupts.CPU120.RES:Rescheduling_interrupts
>>      611457 ±  2%     +10.2%     673608 ±  2%  
>> interrupts.CPU121.LOC:Local_timer_interrupts
>>        7376 ±  3%     +15.6%       8525 ±  4%  
>> interrupts.CPU121.RES:Rescheduling_interrupts
>>      611426 ±  2%     +10.2%     673610 ±  2%  
>> interrupts.CPU122.LOC:Local_timer_interrupts
>>        7466 ±  5%     +11.7%       8336 ±  4%  
>> interrupts.CPU122.RES:Rescheduling_interrupts
>>      611384 ±  2%     +10.2%     673577 ±  2%  
>> interrupts.CPU123.LOC:Local_timer_interrupts
>>        7240 ±  5%     +18.2%       8559 ±  3%  
>> interrupts.CPU123.RES:Rescheduling_interrupts
>>      611388 ±  2%     +10.2%     673597 ±  2%  
>> interrupts.CPU124.LOC:Local_timer_interrupts
>>        7245 ±  5%     +18.0%       8548 ±  5%  
>> interrupts.CPU124.RES:Rescheduling_interrupts
>>      611391 ±  2%     +10.2%     673567 ±  2%  
>> interrupts.CPU125.LOC:Local_timer_interrupts
>>        7318 ±  3%     +15.3%       8436 ±  5%  
>> interrupts.CPU125.RES:Rescheduling_interrupts
>>      611421 ±  2%     +10.2%     673570 ±  2%  
>> interrupts.CPU126.LOC:Local_timer_interrupts
>>        7258 ±  5%     +16.4%       8448 ±  4%  
>> interrupts.CPU126.RES:Rescheduling_interrupts
>>      611468 ±  2%     +10.2%     673614 ±  2%  
>> interrupts.CPU127.LOC:Local_timer_interrupts
>>        7276 ±  5%     +16.9%       8508 ±  5%  
>> interrupts.CPU127.RES:Rescheduling_interrupts
>>      611422 ±  2%     +10.2%     673591 ±  2%  
>> interrupts.CPU128.LOC:Local_timer_interrupts
>>        7278 ±  4%     +17.0%       8518 ±  3%  
>> interrupts.CPU128.RES:Rescheduling_interrupts
>>      611407 ±  2%     +10.2%     673562 ±  2%  
>> interrupts.CPU129.LOC:Local_timer_interrupts
>>        7284 ±  3%     +17.0%       8519 ±  5%  
>> interrupts.CPU129.RES:Rescheduling_interrupts
>>      611825 ±  2%     +10.1%     673920 ±  2%  
>> interrupts.CPU13.LOC:Local_timer_interrupts
>>        7550 ±  4%     +16.2%       8771 ±  3%  
>> interrupts.CPU13.RES:Rescheduling_interrupts
>>      611428 ±  2%     +10.2%     673571 ±  2%  
>> interrupts.CPU130.LOC:Local_timer_interrupts
>>        7337 ±  5%     +16.4%       8541 ±  5%  
>> interrupts.CPU130.RES:Rescheduling_interrupts
>>      611406 ±  2%     +10.2%     673582 ±  2%  
>> interrupts.CPU131.LOC:Local_timer_interrupts
>>        7205 ±  5%     +17.1%       8440 ±  6%  
>> interrupts.CPU131.RES:Rescheduling_interrupts
>>      611449 ±  2%     +10.2%     673586 ±  2%  
>> interrupts.CPU132.LOC:Local_timer_interrupts
>>        7431 ±  4%     +12.7%       8372 ±  3%  
>> interrupts.CPU132.RES:Rescheduling_interrupts
>>      611464 ±  2%     +10.2%     673588 ±  2%  
>> interrupts.CPU133.LOC:Local_timer_interrupts
>>        7289 ±  4%     +15.9%       8446 ±  3%  
>> interrupts.CPU133.RES:Rescheduling_interrupts
>>      611437 ±  2%     +10.2%     673583 ±  2%  
>> interrupts.CPU134.LOC:Local_timer_interrupts
>>        7321 ±  4%     +17.7%       8616 ±  4%  
>> interrupts.CPU134.RES:Rescheduling_interrupts
>>      611401 ±  2%     +10.2%     673595 ±  2%  
>> interrupts.CPU135.LOC:Local_timer_interrupts
>>        7300 ±  4%     +15.7%       8446 ±  4%  
>> interrupts.CPU135.RES:Rescheduling_interrupts
>>      611462 ±  2%     +10.2%     673602 ±  2%  
>> interrupts.CPU136.LOC:Local_timer_interrupts
>>        7389 ±  4%     +13.9%       8417 ±  5%  
>> interrupts.CPU136.RES:Rescheduling_interrupts
>>      611424 ±  2%     +10.2%     673588 ±  2%  
>> interrupts.CPU137.LOC:Local_timer_interrupts
>>        7126 ±  4%     +20.0%       8550 ±  5%  
>> interrupts.CPU137.RES:Rescheduling_interrupts
>>      611383 ±  2%     +10.2%     673556 ±  2%  
>> interrupts.CPU138.LOC:Local_timer_interrupts
>>        7297 ±  5%     +16.1%       8475 ±  5%  
>> interrupts.CPU138.RES:Rescheduling_interrupts
>>      611422 ±  2%     +10.2%     673603 ±  2%  
>> interrupts.CPU139.LOC:Local_timer_interrupts
>>        7160 ±  4%     +17.6%       8423 ±  4%  
>> interrupts.CPU139.RES:Rescheduling_interrupts
>>      611815 ±  2%     +10.1%     673901 ±  2%  
>> interrupts.CPU14.LOC:Local_timer_interrupts
>>        7576 ±  5%     +16.3%       8810 ±  3%  
>> interrupts.CPU14.RES:Rescheduling_interrupts
>>      611388 ±  2%     +10.2%     673577 ±  2%  
>> interrupts.CPU140.LOC:Local_timer_interrupts
>>        7342 ±  6%     +12.9%       8293 ±  4%  
>> interrupts.CPU140.RES:Rescheduling_interrupts
>>      611394 ±  2%     +10.2%     673596 ±  2%  
>> interrupts.CPU141.LOC:Local_timer_interrupts
>>        7257 ±  4%     +16.1%       8423 ±  5%  
>> interrupts.CPU141.RES:Rescheduling_interrupts
>>      611411 ±  2%     +10.2%     673590 ±  2%  
>> interrupts.CPU142.LOC:Local_timer_interrupts
>>        7300 ±  6%     +16.1%       8475 ±  3%  
>> interrupts.CPU142.RES:Rescheduling_interrupts
>>      611396 ±  2%     +10.2%     673622 ±  2%  
>> interrupts.CPU143.LOC:Local_timer_interrupts
>>        7399 ±  5%     +14.0%       8435 ±  4%  
>> interrupts.CPU143.RES:Rescheduling_interrupts
>>      610929 ±  2%     +10.3%     673686 ±  2%  
>> interrupts.CPU144.LOC:Local_timer_interrupts
>>        7281 ±  5%     +13.4%       8257 ±  5%  
>> interrupts.CPU144.RES:Rescheduling_interrupts
>>      610918 ±  2%     +10.3%     673755 ±  2%  
>> interrupts.CPU145.LOC:Local_timer_interrupts
>>        7343 ±  5%     +14.5%       8408 ±  4%  
>> interrupts.CPU145.RES:Rescheduling_interrupts
>>      610910 ±  2%     +10.3%     673705 ±  2%  
>> interrupts.CPU146.LOC:Local_timer_interrupts
>>        7524 ±  4%     +13.6%       8550 ±  6%  
>> interrupts.CPU146.RES:Rescheduling_interrupts
>>      610858 ±  2%     +10.3%     673769 ±  2%  
>> interrupts.CPU147.LOC:Local_timer_interrupts
>>        7433 ±  4%     +10.8%       8237 ±  3%  
>> interrupts.CPU147.RES:Rescheduling_interrupts
>>      610872 ±  2%     +10.3%     673727 ±  2%  
>> interrupts.CPU148.LOC:Local_timer_interrupts
>>        7354 ±  3%     +13.1%       8320 ±  4%  
>> interrupts.CPU148.RES:Rescheduling_interrupts
>>      610909 ±  2%     +10.3%     673750 ±  2%  
>> interrupts.CPU149.LOC:Local_timer_interrupts
>>        7365 ±  3%     +16.5%       8582 ±  3%  
>> interrupts.CPU149.RES:Rescheduling_interrupts
>>      611818 ±  2%     +10.2%     673937 ±  2%  
>> interrupts.CPU15.LOC:Local_timer_interrupts
>>        7632 ±  4%     +12.5%       8587 ±  4%  
>> interrupts.CPU15.RES:Rescheduling_interrupts
>>      610898 ±  2%     +10.3%     673770 ±  2%  
>> interrupts.CPU150.LOC:Local_timer_interrupts
>>        7369 ±  4%     +12.4%       8282 ±  3%  
>> interrupts.CPU150.RES:Rescheduling_interrupts
>>      610887 ±  2%     +10.3%     673754 ±  2%  
>> interrupts.CPU151.LOC:Local_timer_interrupts
>>        7322 ±  4%     +15.3%       8445 ±  3%  
>> interrupts.CPU151.RES:Rescheduling_interrupts
>>      610849 ±  2%     +10.3%     673762 ±  2%  
>> interrupts.CPU152.LOC:Local_timer_interrupts
>>        7305 ±  4%     +15.5%       8439 ±  5%  
>> interrupts.CPU152.RES:Rescheduling_interrupts
>>      610909 ±  2%     +10.3%     673759 ±  2%  
>> interrupts.CPU153.LOC:Local_timer_interrupts
>>        7363 ±  6%     +14.5%       8432 ±  2%  
>> interrupts.CPU153.RES:Rescheduling_interrupts
>>      610878 ±  2%     +10.3%     673730 ±  2%  
>> interrupts.CPU154.LOC:Local_timer_interrupts
>>        7373 ±  4%     +14.1%       8414 ±  4%  
>> interrupts.CPU154.RES:Rescheduling_interrupts
>>      610891 ±  2%     +10.3%     673720 ±  2%  
>> interrupts.CPU155.LOC:Local_timer_interrupts
>>        7385 ±  5%     +11.8%       8260 ±  4%  
>> interrupts.CPU155.RES:Rescheduling_interrupts
>>      610909 ±  2%     +10.3%     673753 ±  2%  
>> interrupts.CPU156.LOC:Local_timer_interrupts
>>        7377 ±  5%     +14.0%       8410 ±  5%  
>> interrupts.CPU156.RES:Rescheduling_interrupts
>>      610886 ±  2%     +10.3%     673754 ±  2%  
>> interrupts.CPU157.LOC:Local_timer_interrupts
>>        7399 ±  5%     +13.1%       8370 ±  4%  
>> interrupts.CPU157.RES:Rescheduling_interrupts
>>      610854 ±  2%     +10.3%     673699 ±  2%  
>> interrupts.CPU158.LOC:Local_timer_interrupts
>>        7489 ±  4%     +10.5%       8278 ±  2%  
>> interrupts.CPU158.RES:Rescheduling_interrupts
>>      610921 ±  2%     +10.3%     673726 ±  2%  
>> interrupts.CPU159.LOC:Local_timer_interrupts
>>        7281 ±  5%     +15.4%       8402 ±  4%  
>> interrupts.CPU159.RES:Rescheduling_interrupts
>>      611836 ±  2%     +10.1%     673889 ±  2%  
>> interrupts.CPU16.LOC:Local_timer_interrupts
>>        7548 ±  4%     +15.6%       8723 ±  5%  
>> interrupts.CPU16.RES:Rescheduling_interrupts
>>      610870 ±  2%     +10.3%     673742 ±  2%  
>> interrupts.CPU160.LOC:Local_timer_interrupts
>>      610856 ±  2%     +10.3%     673722 ±  2%  
>> interrupts.CPU161.LOC:Local_timer_interrupts
>>        7472 ±  5%     +13.4%       8475 ±  5%  
>> interrupts.CPU161.RES:Rescheduling_interrupts
>>      610891 ±  2%     +10.3%     673708 ±  2%  
>> interrupts.CPU162.LOC:Local_timer_interrupts
>>        7177 ±  5%     +14.8%       8239 ±  5%  
>> interrupts.CPU162.RES:Rescheduling_interrupts
>>      610864 ±  2%     +10.3%     673696 ±  2%  
>> interrupts.CPU163.LOC:Local_timer_interrupts
>>        7337 ±  5%     +12.7%       8266 ±  4%  
>> interrupts.CPU163.RES:Rescheduling_interrupts
>>      610858 ±  2%     +10.3%     673724 ±  2%  
>> interrupts.CPU164.LOC:Local_timer_interrupts
>>        7269 ±  5%     +15.2%       8376 ±  4%  
>> interrupts.CPU164.RES:Rescheduling_interrupts
>>      610861 ±  2%     +10.3%     673711 ±  2%  
>> interrupts.CPU165.LOC:Local_timer_interrupts
>>        7431 ±  4%      +9.4%       8131 ±  4%  
>> interrupts.CPU165.RES:Rescheduling_interrupts
>>      610867 ±  2%     +10.3%     673684 ±  2%  
>> interrupts.CPU166.LOC:Local_timer_interrupts
>>        7339 ±  2%     +14.1%       8374 ±  4%  
>> interrupts.CPU166.RES:Rescheduling_interrupts
>>      610913 ±  2%     +10.3%     673730 ±  2%  
>> interrupts.CPU167.LOC:Local_timer_interrupts
>>        7174 ±  3%     +17.5%       8427 ±  4%  
>> interrupts.CPU167.RES:Rescheduling_interrupts
>>      611289 ±  2%     +10.2%     673888 ±  2%  
>> interrupts.CPU168.LOC:Local_timer_interrupts
>>        7311 ±  5%     +14.7%       8382 ±  3%  
>> interrupts.CPU168.RES:Rescheduling_interrupts
>>      611333 ±  2%     +10.2%     673887 ±  2%  
>> interrupts.CPU169.LOC:Local_timer_interrupts
>>        7232 ±  2%     +17.8%       8517 ±  5%  
>> interrupts.CPU169.RES:Rescheduling_interrupts
>>      611838 ±  2%     +10.1%     673879 ±  2%  
>> interrupts.CPU17.LOC:Local_timer_interrupts
>>        7539 ±  2%     +17.7%       8873 ± 10%  
>> interrupts.CPU17.RES:Rescheduling_interrupts
>>      611349 ±  2%     +10.2%     673889 ±  2%  
>> interrupts.CPU170.LOC:Local_timer_interrupts
>>        7374 ±  2%     +14.2%       8424 ±  5%  
>> interrupts.CPU170.RES:Rescheduling_interrupts
>>      611297 ±  2%     +10.2%     673873 ±  2%  
>> interrupts.CPU171.LOC:Local_timer_interrupts
>>        7438 ±  4%     +13.9%       8471 ±  5%  
>> interrupts.CPU171.RES:Rescheduling_interrupts
>>      611304 ±  2%     +10.2%     673845 ±  2%  
>> interrupts.CPU172.LOC:Local_timer_interrupts
>>        7317 ±  3%     +16.6%       8529 ±  5%  
>> interrupts.CPU172.RES:Rescheduling_interrupts
>>      611297 ±  2%     +10.2%     673887 ±  2%  
>> interrupts.CPU173.LOC:Local_timer_interrupts
>>      611309 ±  2%     +10.2%     673858 ±  2%  
>> interrupts.CPU174.LOC:Local_timer_interrupts
>>        7369 ±  5%     +15.8%       8531 ±  5%  
>> interrupts.CPU174.RES:Rescheduling_interrupts
>>      611311 ±  2%     +10.2%     673872 ±  2%  
>> interrupts.CPU175.LOC:Local_timer_interrupts
>>        7352 ±  4%     +15.1%       8465 ±  4%  
>> interrupts.CPU175.RES:Rescheduling_interrupts
>>      611309 ±  2%     +10.2%     673886 ±  2%  
>> interrupts.CPU176.LOC:Local_timer_interrupts
>>        7395 ±  2%     +14.0%       8434 ±  5%  
>> interrupts.CPU176.RES:Rescheduling_interrupts
>>      611322 ±  2%     +10.2%     673868 ±  2%  
>> interrupts.CPU177.LOC:Local_timer_interrupts
>>      611248 ±  2%     +10.2%     673805 ±  2%  
>> interrupts.CPU178.LOC:Local_timer_interrupts
>>        7355 ±  5%     +14.1%       8393 ±  4%  
>> interrupts.CPU178.RES:Rescheduling_interrupts
>>      611319 ±  2%     +10.2%     673906 ±  2%  
>> interrupts.CPU179.LOC:Local_timer_interrupts
>>        7269 ±  4%     +15.2%       8374 ±  6%  
>> interrupts.CPU179.RES:Rescheduling_interrupts
>>      611799 ±  2%     +10.1%     673864 ±  2%  
>> interrupts.CPU18.LOC:Local_timer_interrupts
>>        7585 ±  4%     +14.5%       8683 ±  4%  
>> interrupts.CPU18.RES:Rescheduling_interrupts
>>      611330 ±  2%     +10.2%     673869 ±  2%  
>> interrupts.CPU180.LOC:Local_timer_interrupts
>>        7404 ±  4%     +14.0%       8442 ±  5%  
>> interrupts.CPU180.RES:Rescheduling_interrupts
>>      611324 ±  2%     +10.2%     673889 ±  2%  
>> interrupts.CPU181.LOC:Local_timer_interrupts
>>        7435 ±  2%     +13.0%       8399 ±  3%  
>> interrupts.CPU181.RES:Rescheduling_interrupts
>>      611322 ±  2%     +10.2%     673874 ±  2%  
>> interrupts.CPU182.LOC:Local_timer_interrupts
>>        7250 ±  4%     +14.6%       8305 ±  4%  
>> interrupts.CPU182.RES:Rescheduling_interrupts
>>      611307 ±  2%     +10.2%     673852 ±  2%  
>> interrupts.CPU183.LOC:Local_timer_interrupts
>>        7320 ±  4%     +13.2%       8290 ±  5%  
>> interrupts.CPU183.RES:Rescheduling_interrupts
>>      611329 ±  2%     +10.2%     673877 ±  2%  
>> interrupts.CPU184.LOC:Local_timer_interrupts
>>        7377 ±  4%     +12.5%       8299 ±  5%  
>> interrupts.CPU184.RES:Rescheduling_interrupts
>>      611312 ±  2%     +10.2%     673866 ±  2%  
>> interrupts.CPU185.LOC:Local_timer_interrupts
>>        7333 ±  4%     +13.9%       8351 ±  5%  
>> interrupts.CPU185.RES:Rescheduling_interrupts
>>      611290 ±  2%     +10.2%     673868 ±  2%  
>> interrupts.CPU186.LOC:Local_timer_interrupts
>>        7426 ±  4%     +14.1%       8471 ±  3%  
>> interrupts.CPU186.RES:Rescheduling_interrupts
>>      611304 ±  2%     +10.2%     673853 ±  2%  
>> interrupts.CPU187.LOC:Local_timer_interrupts
>>        7380 ±  3%     +12.5%       8304 ±  5%  
>> interrupts.CPU187.RES:Rescheduling_interrupts
>>      611307 ±  2%     +10.2%     673881 ±  2%  
>> interrupts.CPU188.LOC:Local_timer_interrupts
>>        7277 ±  5%     +16.3%       8467 ±  6%  
>> interrupts.CPU188.RES:Rescheduling_interrupts
>>      611299 ±  2%     +10.2%     673879 ±  2%  
>> interrupts.CPU189.LOC:Local_timer_interrupts
>>        7436 ±  3%     +14.5%       8514 ±  6%  
>> interrupts.CPU189.RES:Rescheduling_interrupts
>>      611761 ±  2%     +10.2%     673928 ±  2%  
>> interrupts.CPU19.LOC:Local_timer_interrupts
>>        7515 ±  6%     +12.7%       8467 ±  4%  
>> interrupts.CPU19.RES:Rescheduling_interrupts
>>      611321 ±  2%     +10.2%     673856 ±  2%  
>> interrupts.CPU190.LOC:Local_timer_interrupts
>>        7188 ±  3%     +16.4%       8366 ±  5%  
>> interrupts.CPU190.RES:Rescheduling_interrupts
>>      611544 ±  2%     +10.2%     674120 ±  2%  
>> interrupts.CPU191.LOC:Local_timer_interrupts
>>        7207 ±  3%     +15.3%       8312 ±  3%  
>> interrupts.CPU191.RES:Rescheduling_interrupts
>>      611805 ±  2%     +10.2%     673920 ±  2%  
>> interrupts.CPU2.LOC:Local_timer_interrupts
>>      611781 ±  2%     +10.2%     673897 ±  2%  
>> interrupts.CPU20.LOC:Local_timer_interrupts
>>        7531 ±  5%     +16.4%       8768 ±  3%  
>> interrupts.CPU20.RES:Rescheduling_interrupts
>>      611715 ±  2%     +10.2%     673897 ±  2%  
>> interrupts.CPU21.LOC:Local_timer_interrupts
>>        7467 ±  3%     +14.9%       8577 ±  4%  
>> interrupts.CPU21.RES:Rescheduling_interrupts
>>      611834 ±  2%     +10.1%     673900 ±  2%  
>> interrupts.CPU22.LOC:Local_timer_interrupts
>>        7427 ±  6%     +17.9%       8754 ±  4%  
>> interrupts.CPU22.RES:Rescheduling_interrupts
>>      611788 ±  2%     +10.1%     673869 ±  2%  
>> interrupts.CPU23.LOC:Local_timer_interrupts
>>        7592 ±  4%     +12.2%       8518 ±  4%  
>> interrupts.CPU23.RES:Rescheduling_interrupts
>>      611426 ±  2%     +10.2%     673619 ±  2%  
>> interrupts.CPU24.LOC:Local_timer_interrupts
>>        7457 ±  4%     +14.7%       8551 ±  3%  
>> interrupts.CPU24.RES:Rescheduling_interrupts
>>      611439 ±  2%     +10.2%     673636 ±  2%  
>> interrupts.CPU25.LOC:Local_timer_interrupts
>>      611402 ±  2%     +10.2%     673620 ±  2%  
>> interrupts.CPU26.LOC:Local_timer_interrupts
>>        7463 ±  3%     +18.4%       8838 ±  7%  
>> interrupts.CPU26.RES:Rescheduling_interrupts
>>      611413 ±  2%     +10.2%     673615 ±  2%  
>> interrupts.CPU27.LOC:Local_timer_interrupts
>>        7412 ±  4%     +16.1%       8607 ±  4%  
>> interrupts.CPU27.RES:Rescheduling_interrupts
>>      611409 ±  2%     +10.2%     673625 ±  2%  
>> interrupts.CPU28.LOC:Local_timer_interrupts
>>        7478 ±  5%     +15.6%       8648 ±  3%  
>> interrupts.CPU28.RES:Rescheduling_interrupts
>>      611415 ±  2%     +10.2%     673620 ±  2%  
>> interrupts.CPU29.LOC:Local_timer_interrupts
>>        7431 ±  2%     +15.5%       8585 ±  2%  
>> interrupts.CPU29.RES:Rescheduling_interrupts
>>      611807 ±  2%     +10.1%     673896 ±  2%  
>> interrupts.CPU3.LOC:Local_timer_interrupts
>>        7659 ±  5%     +20.9%       9264 ± 10%  
>> interrupts.CPU3.RES:Rescheduling_interrupts
>>      611433 ±  2%     +10.2%     673520 ±  2%  
>> interrupts.CPU30.LOC:Local_timer_interrupts
>>        7432 ±  3%     +16.5%       8659 ±  4%  
>> interrupts.CPU30.RES:Rescheduling_interrupts
>>      611450 ±  2%     +10.2%     673604 ±  2%  
>> interrupts.CPU31.LOC:Local_timer_interrupts
>>        7490 ±  4%     +14.6%       8585 ±  5%  
>> interrupts.CPU31.RES:Rescheduling_interrupts
>>      611444 ±  2%     +10.2%     673588 ±  2%  
>> interrupts.CPU32.LOC:Local_timer_interrupts
>>        7393 ±  4%     +19.0%       8799 ±  3%  
>> interrupts.CPU32.RES:Rescheduling_interrupts
>>      611407 ±  2%     +10.2%     673606 ±  2%  
>> interrupts.CPU33.LOC:Local_timer_interrupts
>>        7398 ±  4%     +16.8%       8644 ±  4%  
>> interrupts.CPU33.RES:Rescheduling_interrupts
>>      611452 ±  2%     +10.2%     673609 ±  2%  
>> interrupts.CPU34.LOC:Local_timer_interrupts
>>        7414 ±  5%     +18.8%       8806 ±  4%  
>> interrupts.CPU34.RES:Rescheduling_interrupts
>>      611433 ±  2%     +10.2%     673588 ±  2%  
>> interrupts.CPU35.LOC:Local_timer_interrupts
>>        7486 ±  5%     +18.6%       8878 ±  3%  
>> interrupts.CPU35.RES:Rescheduling_interrupts
>>      611450 ±  2%     +10.2%     673603 ±  2%  
>> interrupts.CPU36.LOC:Local_timer_interrupts
>>        7503 ±  3%     +15.0%       8627 ±  3%  
>> interrupts.CPU36.RES:Rescheduling_interrupts
>>      611465 ±  2%     +10.2%     673611 ±  2%  
>> interrupts.CPU37.LOC:Local_timer_interrupts
>>        7416 ±  3%     +17.6%       8719 ±  2%  
>> interrupts.CPU37.RES:Rescheduling_interrupts
>>      611445 ±  2%     +10.2%     673575 ±  2%  
>> interrupts.CPU38.LOC:Local_timer_interrupts
>>        7477 ±  5%     +14.2%       8540 ±  3%  
>> interrupts.CPU38.RES:Rescheduling_interrupts
>>      611420 ±  2%     +10.2%     673600 ±  2%  
>> interrupts.CPU39.LOC:Local_timer_interrupts
>>        7562 ±  5%     +14.0%       8620 ±  4%  
>> interrupts.CPU39.RES:Rescheduling_interrupts
>>      611823 ±  2%     +10.2%     673949 ±  2%  
>> interrupts.CPU4.LOC:Local_timer_interrupts
>>        7579 ±  4%     +14.9%       8708 ±  3%  
>> interrupts.CPU4.RES:Rescheduling_interrupts
>>      611390 ±  2%     +10.2%     673618 ±  2%  
>> interrupts.CPU40.LOC:Local_timer_interrupts
>>        7444 ±  4%     +15.9%       8626 ±  3%  
>> interrupts.CPU40.RES:Rescheduling_interrupts
>>      611429 ±  2%     +10.2%     673612 ±  2%  
>> interrupts.CPU41.LOC:Local_timer_interrupts
>>        7464 ±  5%     +15.3%       8605 ±  2%  
>> interrupts.CPU41.RES:Rescheduling_interrupts
>>      611418 ±  2%     +10.2%     673597 ±  2%  
>> interrupts.CPU42.LOC:Local_timer_interrupts
>>        7394 ±  4%     +18.6%       8772 ±  5%  
>> interrupts.CPU42.RES:Rescheduling_interrupts
>>      611364 ±  2%     +10.2%     673601 ±  2%  
>> interrupts.CPU43.LOC:Local_timer_interrupts
>>        7483 ±  5%     +15.3%       8630 ±  3%  
>> interrupts.CPU43.RES:Rescheduling_interrupts
>>      611421 ±  2%     +10.2%     673593 ±  2%  
>> interrupts.CPU44.LOC:Local_timer_interrupts
>>        7463 ±  2%     +16.0%       8656 ±  4%  
>> interrupts.CPU44.RES:Rescheduling_interrupts
>>      611423 ±  2%     +10.2%     673593 ±  2%  
>> interrupts.CPU45.LOC:Local_timer_interrupts
>>        7455 ±  4%     +14.8%       8558 ±  3%  
>> interrupts.CPU45.RES:Rescheduling_interrupts
>>      611333 ±  2%     +10.2%     673616 ±  2%  
>> interrupts.CPU46.LOC:Local_timer_interrupts
>>        7406 ±  3%     +17.4%       8696 ±  3%  
>> interrupts.CPU46.RES:Rescheduling_interrupts
>>      611421 ±  2%     +10.2%     673605 ±  2%  
>> interrupts.CPU47.LOC:Local_timer_interrupts
>>        7482 ±  5%     +15.7%       8660 ±  2%  
>> interrupts.CPU47.RES:Rescheduling_interrupts
>>      610910 ±  2%     +10.3%     673737 ±  2%  
>> interrupts.CPU48.LOC:Local_timer_interrupts
>>        7521 ±  6%     +15.1%       8655 ±  4%  
>> interrupts.CPU48.RES:Rescheduling_interrupts
>>      610921 ±  2%     +10.3%     673770 ±  2%  
>> interrupts.CPU49.LOC:Local_timer_interrupts
>>        7594 ±  5%     +12.7%       8558 ±  3%  
>> interrupts.CPU49.RES:Rescheduling_interrupts
>>      611796 ±  2%     +10.1%     673831 ±  2%  
>> interrupts.CPU5.LOC:Local_timer_interrupts
>>      610930 ±  2%     +10.3%     673671 ±  2%  
>> interrupts.CPU50.LOC:Local_timer_interrupts
>>        7504 ±  2%     +17.6%       8824 ±  4%  
>> interrupts.CPU50.RES:Rescheduling_interrupts
>>      610882 ±  2%     +10.3%     673738 ±  2%  
>> interrupts.CPU51.LOC:Local_timer_interrupts
>>      610884 ±  2%     +10.3%     673731 ±  2%  
>> interrupts.CPU52.LOC:Local_timer_interrupts
>>        7436 ±  4%     +18.1%       8783 ±  3%  
>> interrupts.CPU52.RES:Rescheduling_interrupts
>>      610885 ±  2%     +10.3%     673774 ±  2%  
>> interrupts.CPU53.LOC:Local_timer_interrupts
>>        7544 ±  4%     +15.3%       8700 ±  4%  
>> interrupts.CPU53.RES:Rescheduling_interrupts
>>      610902 ±  2%     +10.3%     673763 ±  2%  
>> interrupts.CPU54.LOC:Local_timer_interrupts
>>        7534 ±  4%     +15.6%       8709 ±  6%  
>> interrupts.CPU54.RES:Rescheduling_interrupts
>>      610927 ±  2%     +10.3%     673739 ±  2%  
>> interrupts.CPU55.LOC:Local_timer_interrupts
>>      610883 ±  2%     +10.3%     673719 ±  2%  
>> interrupts.CPU56.LOC:Local_timer_interrupts
>>      610921 ±  2%     +10.3%     673689 ±  2%  
>> interrupts.CPU57.LOC:Local_timer_interrupts
>>        7473 ±  5%     +14.8%       8577 ±  4%  
>> interrupts.CPU57.RES:Rescheduling_interrupts
>>      610893 ±  2%     +10.3%     673781 ±  2%  
>> interrupts.CPU58.LOC:Local_timer_interrupts
>>        7407 ±  5%     +16.6%       8635 ±  3%  
>> interrupts.CPU58.RES:Rescheduling_interrupts
>>      610898 ±  2%     +10.3%     673763 ±  2%  
>> interrupts.CPU59.LOC:Local_timer_interrupts
>>        7642 ±  5%     +13.1%       8643 ±  3%  
>> interrupts.CPU59.RES:Rescheduling_interrupts
>>      611823 ±  2%     +10.1%     673914 ±  2%  
>> interrupts.CPU6.LOC:Local_timer_interrupts
>>        7542 ±  3%     +17.3%       8845 ±  3%  
>> interrupts.CPU6.RES:Rescheduling_interrupts
>>      610869 ±  2%     +10.3%     673784 ±  2%  
>> interrupts.CPU60.LOC:Local_timer_interrupts
>>        7535 ±  4%     +14.8%       8647 ±  5%  
>> interrupts.CPU60.RES:Rescheduling_interrupts
>>      610906 ±  2%     +10.3%     673701 ±  2%  
>> interrupts.CPU61.LOC:Local_timer_interrupts
>>        7631 ±  4%     +13.5%       8660 ±  4%  
>> interrupts.CPU61.RES:Rescheduling_interrupts
>>      610863 ±  2%     +10.3%     673705 ±  2%  
>> interrupts.CPU62.LOC:Local_timer_interrupts
>>        7634 ±  4%     +11.8%       8532 ±  4%  
>> interrupts.CPU62.RES:Rescheduling_interrupts
>>      610920 ±  2%     +10.3%     673730 ±  2%  
>> interrupts.CPU63.LOC:Local_timer_interrupts
>>        7549 ±  5%     +14.8%       8663 ±  5%  
>> interrupts.CPU63.RES:Rescheduling_interrupts
>>      610884 ±  2%     +10.3%     673727 ±  2%  
>> interrupts.CPU64.LOC:Local_timer_interrupts
>>        7644 ±  3%     +13.4%       8672 ±  3%  
>> interrupts.CPU64.RES:Rescheduling_interrupts
>>      610853 ±  2%     +10.3%     673708 ±  2%  
>> interrupts.CPU65.LOC:Local_timer_interrupts
>>        7555 ±  5%     +14.8%       8672 ±  3%  
>> interrupts.CPU65.RES:Rescheduling_interrupts
>>      610950 ±  2%     +10.3%     673804 ±  2%  
>> interrupts.CPU66.LOC:Local_timer_interrupts
>>        7457 ±  5%     +16.8%       8710 ±  4%  
>> interrupts.CPU66.RES:Rescheduling_interrupts
>>      610888 ±  2%     +10.3%     673735 ±  2%  
>> interrupts.CPU67.LOC:Local_timer_interrupts
>>        7550 ±  4%     +11.5%       8415 ±  5%  
>> interrupts.CPU67.RES:Rescheduling_interrupts
>>      610891 ±  2%     +10.3%     673727 ±  2%  
>> interrupts.CPU68.LOC:Local_timer_interrupts
>>        7612 ±  4%     +12.1%       8534 ±  3%  
>> interrupts.CPU68.RES:Rescheduling_interrupts
>>      610901 ±  2%     +10.3%     673659 ±  2%  
>> interrupts.CPU69.LOC:Local_timer_interrupts
>>        7493 ±  4%     +14.8%       8601 ±  4%  
>> interrupts.CPU69.RES:Rescheduling_interrupts
>>      611808 ±  2%     +10.1%     673844 ±  2%  
>> interrupts.CPU7.LOC:Local_timer_interrupts
>>        7673 ±  5%     +15.0%       8821 ±  4%  
>> interrupts.CPU7.RES:Rescheduling_interrupts
>>      610906 ±  2%     +10.3%     673733 ±  2%  
>> interrupts.CPU70.LOC:Local_timer_interrupts
>>        7412 ±  3%     +16.9%       8664 ±  4%  
>> interrupts.CPU70.RES:Rescheduling_interrupts
>>      610871 ±  2%     +10.3%     673671 ±  2%  
>> interrupts.CPU71.LOC:Local_timer_interrupts
>>        7519 ±  4%     +13.8%       8557 ±  5%  
>> interrupts.CPU71.RES:Rescheduling_interrupts
>>      611325 ±  2%     +10.2%     673906 ±  2%  
>> interrupts.CPU72.LOC:Local_timer_interrupts
>>        7720 ±  4%      +9.6%       8459 ±  4%  
>> interrupts.CPU72.RES:Rescheduling_interrupts
>>      611328 ±  2%     +10.2%     673908 ±  2%  
>> interrupts.CPU73.LOC:Local_timer_interrupts
>>        7535 ±  4%     +14.6%       8634 ±  4%  
>> interrupts.CPU73.RES:Rescheduling_interrupts
>>      611311 ±  2%     +10.2%     673883 ±  2%  
>> interrupts.CPU74.LOC:Local_timer_interrupts
>>        7526 ±  5%     +14.7%       8635 ±  6%  
>> interrupts.CPU74.RES:Rescheduling_interrupts
>>      611315 ±  2%     +10.2%     673868 ±  2%  
>> interrupts.CPU75.LOC:Local_timer_interrupts
>>        7712 ±  5%      +9.8%       8469 ±  4%  
>> interrupts.CPU75.RES:Rescheduling_interrupts
>>      611304 ±  2%     +10.2%     673860 ±  2%  
>> interrupts.CPU76.LOC:Local_timer_interrupts
>>        7476 ±  5%     +14.2%       8537 ±  5%  
>> interrupts.CPU76.RES:Rescheduling_interrupts
>>      611322 ±  2%     +10.2%     673834 ±  2%  
>> interrupts.CPU77.LOC:Local_timer_interrupts
>>        7670 ±  3%     +12.5%       8630 ±  5%  
>> interrupts.CPU77.RES:Rescheduling_interrupts
>>      611326 ±  2%     +10.2%     673851 ±  2%  
>> interrupts.CPU78.LOC:Local_timer_interrupts
>>        7417 ±  3%     +15.6%       8578 ±  4%  
>> interrupts.CPU78.RES:Rescheduling_interrupts
>>      611328 ±  2%     +10.2%     673820 ±  2%  
>> interrupts.CPU79.LOC:Local_timer_interrupts
>>        7545 ±  4%     +13.9%       8590 ±  4%  
>> interrupts.CPU79.RES:Rescheduling_interrupts
>>      611842 ±  2%     +10.1%     673928 ±  2%  
>> interrupts.CPU8.LOC:Local_timer_interrupts
>>        7673 ±  3%     +14.9%       8818 ±  4%  
>> interrupts.CPU8.RES:Rescheduling_interrupts
>>      611366 ±  2%     +10.2%     673868 ±  2%  
>> interrupts.CPU80.LOC:Local_timer_interrupts
>>        7513 ±  3%     +16.1%       8723 ±  5%  
>> interrupts.CPU80.RES:Rescheduling_interrupts
>>      611322 ±  2%     +10.2%     673860 ±  2%  
>> interrupts.CPU81.LOC:Local_timer_interrupts
>>      611299 ±  2%     +10.2%     673868 ±  2%  
>> interrupts.CPU82.LOC:Local_timer_interrupts
>>        7523 ±  4%     +13.8%       8562 ±  4%  
>> interrupts.CPU82.RES:Rescheduling_interrupts
>>      611320 ±  2%     +10.2%     673727 ±  2%  
>> interrupts.CPU83.LOC:Local_timer_interrupts
>>        7518 ±  5%     +12.4%       8454 ±  5%  
>> interrupts.CPU83.RES:Rescheduling_interrupts
>>      611326 ±  2%     +10.2%     673899 ±  2%  
>> interrupts.CPU84.LOC:Local_timer_interrupts
>>        7452 ±  6%     +12.3%       8365 ±  5%  
>> interrupts.CPU84.RES:Rescheduling_interrupts
>>      611329 ±  2%     +10.2%     673885 ±  2%  
>> interrupts.CPU85.LOC:Local_timer_interrupts
>>        7399 ±  5%     +11.1%       8218 ±  5%  
>> interrupts.CPU85.RES:Rescheduling_interrupts
>>      611278 ±  2%     +10.2%     673875 ±  2%  
>> interrupts.CPU86.LOC:Local_timer_interrupts
>>        7376 ±  4%     +16.3%       8581 ±  3%  
>> interrupts.CPU86.RES:Rescheduling_interrupts
>>      611313 ±  2%     +10.2%     673874 ±  2%  
>> interrupts.CPU87.LOC:Local_timer_interrupts
>>        7544 ±  4%     +12.8%       8508 ±  4%  
>> interrupts.CPU87.RES:Rescheduling_interrupts
>>      611351 ±  2%     +10.2%     673928 ±  2%  
>> interrupts.CPU88.LOC:Local_timer_interrupts
>>        7418 ±  6%     +13.3%       8406 ±  4%  
>> interrupts.CPU88.RES:Rescheduling_interrupts
>>      611324 ±  2%     +10.2%     673795 ±  2%  
>> interrupts.CPU89.LOC:Local_timer_interrupts
>>        7426 ±  4%     +13.6%       8439 ±  5%  
>> interrupts.CPU89.RES:Rescheduling_interrupts
>>      611832 ±  2%     +10.2%     673968 ±  2%  
>> interrupts.CPU9.LOC:Local_timer_interrupts
>>        7538 ±  3%     +15.6%       8715 ±  4%  
>> interrupts.CPU9.RES:Rescheduling_interrupts
>>      611298 ±  2%     +10.2%     673878 ±  2%  
>> interrupts.CPU90.LOC:Local_timer_interrupts
>>        7421 ±  3%     +12.7%       8363 ±  4%  
>> interrupts.CPU90.RES:Rescheduling_interrupts
>>      611305 ±  2%     +10.2%     673874 ±  2%  
>> interrupts.CPU91.LOC:Local_timer_interrupts
>>        7377 ±  5%     +15.6%       8528 ±  5%  
>> interrupts.CPU91.RES:Rescheduling_interrupts
>>      611288 ±  2%     +10.2%     673876 ±  2%  
>> interrupts.CPU92.LOC:Local_timer_interrupts
>>        7349 ±  4%     +12.0%       8234 ±  4%  
>> interrupts.CPU92.RES:Rescheduling_interrupts
>>      611329 ±  2%     +10.2%     673857 ±  2%  
>> interrupts.CPU93.LOC:Local_timer_interrupts
>>        7438 ±  3%     +14.7%       8530 ±  4%  
>> interrupts.CPU93.RES:Rescheduling_interrupts
>>      611276 ±  2%     +10.2%     673866 ±  2%  
>> interrupts.CPU94.LOC:Local_timer_interrupts
>>        7363 ±  4%     +12.7%       8300 ±  5%  
>> interrupts.CPU94.RES:Rescheduling_interrupts
>>      611381 ±  2%     +10.2%     673922 ±  2%  
>> interrupts.CPU95.LOC:Local_timer_interrupts
>>       10549 ±  3%     +14.9%      12117 ±  4%  
>> interrupts.CPU95.RES:Rescheduling_interrupts
>>      611824 ±  2%     +10.1%     673847 ±  2%  
>> interrupts.CPU96.LOC:Local_timer_interrupts
>>        7416 ±  3%     +12.9%       8375 ±  5%  
>> interrupts.CPU96.RES:Rescheduling_interrupts
>>      611751 ±  2%     +10.2%     673870 ±  2%  
>> interrupts.CPU97.LOC:Local_timer_interrupts
>>        7437 ±  4%     +14.0%       8477 ±  4%  
>> interrupts.CPU97.RES:Rescheduling_interrupts
>>      611787 ±  2%     +10.1%     673877 ±  2%  
>> interrupts.CPU98.LOC:Local_timer_interrupts
>>        7555 ±  4%     +13.0%       8534 ±  3%  
>> interrupts.CPU98.RES:Rescheduling_interrupts
>>      611783 ±  2%     +10.1%     673872 ±  2%  
>> interrupts.CPU99.LOC:Local_timer_interrupts
>>        7452 ±  3%     +12.8%       8404 ±  5%  
>> interrupts.CPU99.RES:Rescheduling_interrupts
>>   1.174e+08 ±  2%     +10.2%  1.294e+08 ±  2%  
>> interrupts.LOC:Local_timer_interrupts
>>     1433875 ±  3%     +14.6%    1643307 ±  3%  
>> interrupts.RES:Rescheduling_interrupts
>>
>>
>>                                    aim7.jobs-per-min
>>    30500 
>> +-------------------------------------------------------------------+
>>          |                               
>> ..+....                            .|
>>    30000 |-+                       ......       
>> ...                      ... |
>>    29500 |................   ......                ..                 
>> ...    |
>>          |                +..                        ...            
>> ..       |
>>    29000 |-+                                            ...      
>> ...         |
>>    28500 |-+                                               .  
>> ...            |
>>          |                                                  
>> +.               |
>>    28000 |-+              O                                 
>> O                |
>>    27500 
>> |-+                                                                 |
>>          
>> |                                                                   |
>>    27000 
>> |-+                                                                 |
>>    26500 
>> |-+                                                                 |
>>          |                                 
>> O                                 |
>>    26000 
>> +-------------------------------------------------------------------+
>> [*] bisect-good sample
>> [O] bisect-bad  sample
>>
>>
>>
>> Disclaimer:
>> Results have been estimated based on internal Intel analysis and are 
>> provided
>> for informational purposes only. Any difference in system hardware or 
>> software
>> design or configuration may affect actual performance.
>>
>>
>> Thanks,
>> Rong Chen
>>
>>
>> _______________________________________________
>> LKP mailing list -- lkp@...ts.01.org
>> To unsubscribe send an email to lkp-leave@...ts.01.org
>>
> 

-- 
Zhengjun Xing

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ