lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <93783add-bce3-2c3b-f24a-642a5a479938@linux.intel.com>
Date:   Tue, 3 Nov 2020 14:04:58 +0800
From:   Xing Zhengjun <zhengjun.xing@...ux.intel.com>
To:     kernel test robot <rong.a.chen@...el.com>,
        Josef Bacik <josef@...icpanda.com>
Cc:     David Sterba <dsterba@...e.com>,
        LKML <linux-kernel@...r.kernel.org>, lkp@...ts.01.org
Subject: Re: [LKP] Re: [btrfs] c75e839414: aim7.jobs-per-min -9.1% regression

Hi Josef,

      I re-test it in v5.10-rc2, the regression still existed. Do you 
have time to take a look at this? Thanks.

On 10/13/2020 2:30 PM, Xing Zhengjun wrote:
> Hi Josef,
> 
>     I re-test in v5.9, the regression still existed. Do you have time to 
> take a look at this? Thanks.
> 
> On 6/15/2020 11:21 AM, Xing Zhengjun wrote:
>> Hi Josef,
>>
>>     Do you have time to take a look at this? Thanks.
>>
>> On 6/12/2020 2:11 PM, kernel test robot wrote:
>>> Greeting,
>>>
>>> FYI, we noticed a -9.1% regression of aim7.jobs-per-min due to commit:
>>>
>>>
>>> commit: c75e839414d3610e6487ae3145199c500d55f7f7 ("btrfs: kill the 
>>> subvol_srcu")
>>> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
>>>
>>> in testcase: aim7
>>> on test machine: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 
>>> 2.30GHz with 192G memory
>>> with following parameters:
>>>
>>>     disk: 4BRD_12G
>>>     md: RAID0
>>>     fs: btrfs
>>>     test: disk_wrt
>>>     load: 1500
>>>     cpufreq_governor: performance
>>>     ucode: 0x500002c
>>>
>>> test-description: AIM7 is a traditional UNIX system level benchmark 
>>> suite which is used to test and measure the performance of multiuser 
>>> system.
>>> test-url: https://sourceforge.net/projects/aimbench/files/aim-suite7/
>>>
>>>
>>>
>>> If you fix the issue, kindly add following tag
>>> Reported-by: kernel test robot <rong.a.chen@...el.com>
>>>
>>>
>>> Details are as below:
>>> --------------------------------------------------------------------------------------------------> 
>>>
>>>
>>>
>>> To reproduce:
>>>
>>>          git clone https://github.com/intel/lkp-tests.git
>>>          cd lkp-tests
>>>          bin/lkp install job.yaml  # job file is attached in this email
>>>          bin/lkp run     job.yaml
>>>
>>> ========================================================================================= 
>>>
>>> compiler/cpufreq_governor/disk/fs/kconfig/load/md/rootfs/tbox_group/test/testcase/ucode: 
>>>
>>> gcc-7/performance/4BRD_12G/btrfs/x86_64-rhel-7.6/1500/RAID0/debian-x86_64-20191114.cgz/lkp-csl-2ap2/disk_wrt/aim7/0x500002c 
>>>
>>>
>>> commit:
>>>    efc3453494 ("btrfs: make btrfs_cleanup_fs_roots use the radix tree 
>>> lock")
>>>    c75e839414 ("btrfs: kill the subvol_srcu")
>>>
>>> efc3453494af7818 c75e839414d3610e6487ae31451
>>> ---------------- ---------------------------
>>>         fail:runs  %reproduction    fail:runs
>>>             |             |             |
>>>            3:9          -33%            :8 
>>> dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x
>>>           %stddev     %change         %stddev
>>>               \          |                \
>>>       29509 ±  2%      -9.1%      26837 ±  2%  aim7.jobs-per-min
>>>      305.28 ±  2%     +10.0%     335.72 ±  2%  aim7.time.elapsed_time
>>>      305.28 ±  2%     +10.0%     335.72 ±  2%  
>>> aim7.time.elapsed_time.max
>>>     4883135 ± 10%     +37.9%    6735464 ±  7% 
>>> aim7.time.involuntary_context_switches
>>>       56288 ±  2%     +10.5%      62202 ±  2%  aim7.time.system_time
>>>     2344783            +6.5%    2497364 ±  2% 
>>> aim7.time.voluntary_context_switches
>>>    62337721 ±  2%      +9.8%   68456490 ±  2%  turbostat.IRQ
>>>      431.56 ±  6%     +22.3%     527.88 ±  4%  vmstat.procs.r
>>>       27340 ±  2%     +11.2%      30397 ±  2%  vmstat.system.cs
>>>      226804 ±  6%     +21.7%     276057 ±  4%  meminfo.Active(file)
>>>      221309 ±  6%     +22.3%     270668 ±  4%  meminfo.Dirty
>>>      720.89 ±111%     +49.3%       1076 ± 73%  meminfo.Mlocked
>>>       14278 ±  2%      -8.3%      13094 ±  2%  meminfo.max_used_kB
>>>       57228 ±  6%     +22.7%      70195 ±  5% 
>>> numa-meminfo.node0.Active(file)
>>>       55433 ±  6%     +21.6%      67431 ±  4%  numa-meminfo.node0.Dirty
>>>       56152 ±  6%     +21.4%      68180 ±  5% 
>>> numa-meminfo.node1.Active(file)
>>>       55001 ±  6%     +22.5%      67397 ±  4%  numa-meminfo.node1.Dirty
>>>       56373 ±  6%     +21.7%      68594 ±  4% 
>>> numa-meminfo.node2.Active(file)
>>>       55222 ±  7%     +22.6%      67726 ±  4%  numa-meminfo.node2.Dirty
>>>       56671 ±  6%     +20.5%      68317 ±  3% 
>>> numa-meminfo.node3.Active(file)
>>>       55285 ±  6%     +21.8%      67355 ±  4%  numa-meminfo.node3.Dirty
>>>       56694 ±  6%     +21.7%      69019 ±  4%  
>>> proc-vmstat.nr_active_file
>>>       55342 ±  6%     +22.3%      67662 ±  4%  proc-vmstat.nr_dirty
>>>      402316            +2.1%     410951        proc-vmstat.nr_file_pages
>>>      180.22 ±111%     +49.4%     269.25 ± 73%  proc-vmstat.nr_mlock
>>>       56694 ±  6%     +21.7%      69019 ±  4% 
>>> proc-vmstat.nr_zone_active_file
>>>       54680 ±  6%     +22.8%      67168 ±  4% 
>>> proc-vmstat.nr_zone_write_pending
>>>     3144381 ±  2%      +6.1%    3335275        proc-vmstat.pgactivate
>>>     1387558 ±  2%      +7.9%    1496754 ±  2%  proc-vmstat.pgfault
>>>      983.33 ±  4%      +5.4%       1036 
>>> proc-vmstat.unevictable_pgs_culled
>>>       14331 ±  6%     +22.6%      17566 ±  5% 
>>> numa-vmstat.node0.nr_active_file
>>>       13884 ±  6%     +21.6%      16884 ±  4%  
>>> numa-vmstat.node0.nr_dirty
>>>       14330 ±  6%     +22.6%      17566 ±  5% 
>>> numa-vmstat.node0.nr_zone_active_file
>>>       13714 ±  6%     +22.2%      16755 ±  4% 
>>> numa-vmstat.node0.nr_zone_write_pending
>>>       14047 ±  6%     +21.3%      17043 ±  4% 
>>> numa-vmstat.node1.nr_active_file
>>>       13763 ±  6%     +22.3%      16838 ±  4%  
>>> numa-vmstat.node1.nr_dirty
>>>       14047 ±  6%     +21.3%      17043 ±  4% 
>>> numa-vmstat.node1.nr_zone_active_file
>>>       13599 ±  6%     +23.0%      16726 ±  4% 
>>> numa-vmstat.node1.nr_zone_write_pending
>>>       14074 ±  5%     +21.7%      17130 ±  4% 
>>> numa-vmstat.node2.nr_active_file
>>>       13786 ±  6%     +22.6%      16906 ±  4%  
>>> numa-vmstat.node2.nr_dirty
>>>       14074 ±  5%     +21.7%      17130 ±  4% 
>>> numa-vmstat.node2.nr_zone_active_file
>>>       13617 ±  7%     +23.3%      16789 ±  4% 
>>> numa-vmstat.node2.nr_zone_write_pending
>>>       14151 ±  6%     +20.7%      17074 ±  3% 
>>> numa-vmstat.node3.nr_active_file
>>>       13790 ±  6%     +22.1%      16832 ±  4%  
>>> numa-vmstat.node3.nr_dirty
>>>       14152 ±  6%     +20.7%      17074 ±  3% 
>>> numa-vmstat.node3.nr_zone_active_file
>>>       13627 ±  6%     +22.5%      16696 ±  4% 
>>> numa-vmstat.node3.nr_zone_write_pending
>>>      828631 ± 16%     +49.9%    1241934 ±  8% 
>>> sched_debug.cfs_rq:/.min_vruntime.stddev
>>>        1.33 ± 30%     +96.9%       2.61 ± 14% 
>>> sched_debug.cfs_rq:/.nr_spread_over.avg
>>>      831152 ± 16%     +49.7%    1244586 ±  8% 
>>> sched_debug.cfs_rq:/.spread0.stddev
>>>        1.92 ± 14%     +28.4%       2.47 ±  8% 
>>> sched_debug.cpu.nr_running.avg
>>>        5.00 ±  9%     +29.7%       6.48 ±  9% 
>>> sched_debug.cpu.nr_running.max
>>>        0.90 ±  9%     +16.0%       1.05 ±  3% 
>>> sched_debug.cpu.nr_running.stddev
>>>       21790 ± 11%     +16.7%      25440 ±  2% 
>>> sched_debug.cpu.nr_switches.avg
>>>       19761 ± 11%     +17.9%      23289 ±  2% 
>>> sched_debug.cpu.nr_switches.min
>>>       20403 ± 12%     +18.1%      24101 ±  2% 
>>> sched_debug.cpu.sched_count.avg
>>>       27825 ± 10%     +15.8%      32227 ±  4% 
>>> sched_debug.cpu.sched_count.max
>>>       18995 ± 13%     +18.3%      22463 ±  2% 
>>> sched_debug.cpu.sched_count.min
>>>        1143 ± 11%     +15.6%       1322 ±  4% 
>>> sched_debug.cpu.sched_count.stddev
>>>        1404 ±  7%     -31.0%     969.66 ± 11% 
>>> sched_debug.cpu.sched_goidle.avg
>>>        1186 ±  8%     -36.0%     759.25 ± 14% 
>>> sched_debug.cpu.sched_goidle.min
>>>       16821 ± 14%     +20.6%      20284 ±  6% 
>>> sched_debug.cpu.ttwu_count.max
>>>        2790 ± 11%     +21.7%       3395 ±  7% 
>>> sched_debug.cpu.ttwu_count.stddev
>>>      398.19 ± 16%     +28.3%     510.73 ± 10% 
>>> sched_debug.cpu.ttwu_local.stddev
>>>    43238516 ±  5%     +19.0%   51459028 ±  4%  perf-stat.i.cache-misses
>>>    83870573 ±  4%     +13.8%   95436555 ±  4% 
>>> perf-stat.i.cache-references
>>>       27181 ±  2%     +11.1%      30198 ±  2% 
>>> perf-stat.i.context-switches
>>>        3008 ±  2%      -7.1%       2794        
>>> perf-stat.i.cpu-migrations
>>>       12612 ±  5%     -15.6%      10641 ±  3% 
>>> perf-stat.i.cycles-between-cache-misses
>>>     1.1e+09 ±  2%      -7.3%  1.019e+09 ±  2%  perf-stat.i.dTLB-stores
>>>    10684629 ±  2%     +10.8%   11842781 ±  2% 
>>> perf-stat.i.node-load-misses
>>>      641429 ±  2%     +24.4%     798190 ±  2%  perf-stat.i.node-loads
>>>     5915955 ±  2%      -6.3%    5543368 ±  2% 
>>> perf-stat.i.node-store-misses
>>>      380971            -6.7%     355629 ±  2%  perf-stat.i.node-stores
>>>        1.66 ±  4%     +14.4%       1.90 ±  3%  perf-stat.overall.MPKI
>>>       13064 ±  5%     -15.8%      10999 ±  4% 
>>> perf-stat.overall.cycles-between-cache-misses
>>>    43484558 ±  5%     +19.2%   51837476 ±  4%  perf-stat.ps.cache-misses
>>>    83278267 ±  4%     +14.4%   95253129 ±  3% 
>>> perf-stat.ps.cache-references
>>>       27414 ±  2%     +11.2%      30486 ±  2% 
>>> perf-stat.ps.context-switches
>>>        3028 ±  2%      -7.0%       2817 perf-stat.ps.cpu-migrations
>>>   1.104e+09 ±  2%      -7.3%  1.023e+09 ±  2%  perf-stat.ps.dTLB-stores
>>>        4394            -1.8%       4315        perf-stat.ps.minor-faults
>>>    10766873 ±  2%     +11.0%   11947268 ±  2% 
>>> perf-stat.ps.node-load-misses
>>>      645940 ±  2%     +24.7%     805392 ±  2%  perf-stat.ps.node-loads
>>>     5965618 ±  2%      -6.2%    5595562 ±  2% 
>>> perf-stat.ps.node-store-misses
>>>      383092            -6.5%     358124 ±  2%  perf-stat.ps.node-stores
>>>        4395            -1.8%       4315        perf-stat.ps.page-faults
>>>   1.538e+13 ±  2%     +10.0%  1.692e+13 ±  2% 
>>> perf-stat.total.instructions
>>>       27.39            -0.1       27.28 
>>> perf-profile.calltrace.cycles-pp.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 
>>>
>>>       35.13            +0.1       35.23 
>>> perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write 
>>>
>>>       35.23            +0.1       35.34 
>>> perf-profile.calltrace.cycles-pp._raw_spin_lock.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter 
>>>
>>>       35.48            +0.1       35.60 
>>> perf-profile.calltrace.cycles-pp.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write 
>>>
>>>       35.08            +0.1       35.22 
>>> perf-profile.calltrace.cycles-pp._raw_spin_lock.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter 
>>>
>>>       34.97            +0.1       35.12 
>>> perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write 
>>>
>>>       35.57            +0.1       35.72 
>>> perf-profile.calltrace.cycles-pp.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 
>>>
>>>       35.10            +0.2       35.26 
>>> perf-profile.calltrace.cycles-pp.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write 
>>>
>>>       35.11            +0.2       35.27 
>>> perf-profile.calltrace.cycles-pp.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 
>>>
>>>       98.71            +0.2       98.88 
>>> perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 
>>>
>>>       98.69            +0.2       98.87 
>>> perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 
>>>
>>>       98.74            +0.2       98.91 
>>> perf-profile.calltrace.cycles-pp.write
>>>       98.71            +0.2       98.89 
>>> perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write
>>>       98.68            +0.2       98.86 
>>> perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 
>>>
>>>       98.64            +0.2       98.82 
>>> perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 
>>>
>>>       98.63            +0.2       98.81 
>>> perf-profile.calltrace.cycles-pp.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64 
>>>
>>>       98.58            +0.2       98.77 
>>> perf-profile.calltrace.cycles-pp.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write 
>>>
>>>       27.39            -0.1       27.28 
>>> perf-profile.children.cycles-pp.btrfs_dirty_pages
>>>        0.56 ±  5%      -0.1        0.46 ±  9% 
>>> perf-profile.children.cycles-pp.btrfs_get_extent
>>>        0.51 ±  5%      -0.1        0.41 ±  9% 
>>> perf-profile.children.cycles-pp.btrfs_lookup_file_extent
>>>        0.52 ±  5%      -0.1        0.43 ±  9% 
>>> perf-profile.children.cycles-pp.btrfs_search_slot
>>>        0.44 ±  6%      -0.1        0.35 ± 11% 
>>> perf-profile.children.cycles-pp.btrfs_read_lock_root_node
>>>        0.43 ±  6%      -0.1        0.34 ± 11% 
>>> perf-profile.children.cycles-pp.btrfs_tree_read_lock
>>>        0.19 ± 15%      -0.1        0.11 ± 27% 
>>> perf-profile.children.cycles-pp.start_secondary
>>>        0.19 ± 16%      -0.1        0.10 ± 27% 
>>> perf-profile.children.cycles-pp.cpuidle_enter
>>>        0.19 ± 16%      -0.1        0.10 ± 27% 
>>> perf-profile.children.cycles-pp.cpuidle_enter_state
>>>        0.41 ±  7%      -0.1        0.33 ±  8% 
>>> perf-profile.children.cycles-pp.rwsem_optimistic_spin
>>>        0.46 ±  7%      -0.1        0.38 ±  8% 
>>> perf-profile.children.cycles-pp.rwsem_down_write_slowpath
>>>        0.19 ± 15%      -0.1        0.11 ± 25% 
>>> perf-profile.children.cycles-pp.secondary_startup_64
>>>        0.19 ± 15%      -0.1        0.11 ± 25% 
>>> perf-profile.children.cycles-pp.cpu_startup_entry
>>>        0.19 ± 15%      -0.1        0.11 ± 25% 
>>> perf-profile.children.cycles-pp.do_idle
>>>        0.18 ± 15%      -0.1        0.10 ± 24% 
>>> perf-profile.children.cycles-pp.intel_idle
>>>        0.29 ±  9%      -0.1        0.22 ± 11% 
>>> perf-profile.children.cycles-pp.osq_lock
>>>        0.19 ±  6%      -0.0        0.15 ± 16% 
>>> perf-profile.children.cycles-pp.finish_wait
>>>        0.39 ±  5%      -0.0        0.35 ±  6% 
>>> perf-profile.children.cycles-pp.do_unlinkat
>>>        0.39 ±  6%      -0.0        0.35 ±  7% 
>>> perf-profile.children.cycles-pp.unlink
>>>        0.26 ±  5%      -0.0        0.22 ±  6% 
>>> perf-profile.children.cycles-pp.do_sys_open
>>>        0.26 ±  5%      -0.0        0.22 ±  6% 
>>> perf-profile.children.cycles-pp.do_sys_openat2
>>>        0.26 ±  4%      -0.0        0.22 ±  6% 
>>> perf-profile.children.cycles-pp.do_filp_open
>>>        0.26 ±  4%      -0.0        0.22 ±  6% 
>>> perf-profile.children.cycles-pp.path_openat
>>>        0.26 ±  4%      -0.0        0.22 ±  6% 
>>> perf-profile.children.cycles-pp.creat
>>>        0.00            +0.0        0.05 
>>> perf-profile.children.cycles-pp.btrfs_calculate_inode_block_rsv_size
>>>       99.65            +0.1       99.73 
>>> perf-profile.children.cycles-pp.do_syscall_64
>>>       99.66            +0.1       99.74 
>>> perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
>>>       35.62            +0.1       35.74 
>>> perf-profile.children.cycles-pp.btrfs_reserve_metadata_bytes
>>>       35.57            +0.1       35.72 
>>> perf-profile.children.cycles-pp.btrfs_delalloc_reserve_metadata
>>>       97.55            +0.2       97.72 
>>> perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
>>>       98.74            +0.2       98.92 
>>> perf-profile.children.cycles-pp.write
>>>       98.70            +0.2       98.88 
>>> perf-profile.children.cycles-pp.ksys_write
>>>       98.69            +0.2       98.87 
>>> perf-profile.children.cycles-pp.vfs_write
>>>       98.65            +0.2       98.83 
>>> perf-profile.children.cycles-pp.new_sync_write
>>>       98.63            +0.2       98.81 
>>> perf-profile.children.cycles-pp.btrfs_file_write_iter
>>>       98.59            +0.2       98.78 
>>> perf-profile.children.cycles-pp.btrfs_buffered_write
>>>       97.60            +0.2       97.79 
>>> perf-profile.children.cycles-pp._raw_spin_lock
>>>        0.18 ± 15%      -0.1        0.10 ± 24% 
>>> perf-profile.self.cycles-pp.intel_idle
>>>        0.28 ±  9%      -0.1        0.21 ± 12% 
>>> perf-profile.self.cycles-pp.osq_lock
>>>        0.45 ±  2%      -0.0        0.41 ±  2% 
>>> perf-profile.self.cycles-pp._raw_spin_lock
>>>      135788 ±  2%     +13.1%     153639 ±  3%  softirqs.CPU0.RCU
>>>      115547 ±  3%      +8.0%     124768 ±  4%  softirqs.CPU0.TIMER
>>>      135243 ±  3%     +14.0%     154188 ±  2%  softirqs.CPU1.RCU
>>>      136056 ±  3%     +12.4%     152960 ±  3%  softirqs.CPU10.RCU
>>>      134253 ±  3%     +14.2%     153260 ±  3%  softirqs.CPU100.RCU
>>>      134668 ±  4%     +13.7%     153157 ±  2%  softirqs.CPU101.RCU
>>>      109995 ±  3%      +7.9%     118637 ±  3%  softirqs.CPU101.TIMER
>>>      134989 ±  4%     +13.9%     153700 ±  2%  softirqs.CPU102.RCU
>>>      109876 ±  3%      +8.9%     119608 ±  2%  softirqs.CPU102.TIMER
>>>      134868 ±  4%     +13.8%     153427 ±  3%  softirqs.CPU103.RCU
>>>      109933 ±  3%      +8.2%     118946 ±  3%  softirqs.CPU103.TIMER
>>>      134454 ±  3%     +14.0%     153285 ±  3%  softirqs.CPU104.RCU
>>>      109731 ±  2%      +8.4%     118965 ±  3%  softirqs.CPU104.TIMER
>>>      134878 ±  3%     +13.6%     153283 ±  3%  softirqs.CPU105.RCU
>>>      134096 ±  3%     +14.1%     153022 ±  3%  softirqs.CPU106.RCU
>>>      109821 ±  2%      +8.2%     118845 ±  3%  softirqs.CPU106.TIMER
>>>      133905 ±  3%     +15.0%     154004 ±  3%  softirqs.CPU107.RCU
>>>      134117 ±  3%     +14.3%     153232 ±  3%  softirqs.CPU108.RCU
>>>      109545 ±  2%      +8.8%     119182 ±  3%  softirqs.CPU108.TIMER
>>>      134574 ±  3%     +13.8%     153202 ±  3%  softirqs.CPU109.RCU
>>>      109804 ±  3%      +8.2%     118760 ±  3%  softirqs.CPU109.TIMER
>>>      135106 ±  3%     +14.8%     155124 ±  3%  softirqs.CPU11.RCU
>>>      111746 ±  4%      +7.8%     120460 ±  3%  softirqs.CPU11.TIMER
>>>      134482 ±  4%     +14.3%     153687 ±  3%  softirqs.CPU110.RCU
>>>      110314 ±  3%      +8.1%     119241 ±  3%  softirqs.CPU110.TIMER
>>>      134464 ±  3%     +14.3%     153634 ±  3%  softirqs.CPU111.RCU
>>>      110626 ±  3%      +8.3%     119815 ±  3%  softirqs.CPU111.TIMER
>>>      134895 ±  3%     +14.4%     154257 ±  3%  softirqs.CPU112.RCU
>>>      110040 ±  3%     +14.6%     126058 ± 16%  softirqs.CPU112.TIMER
>>>      134644 ±  3%     +13.6%     152993 ±  3%  softirqs.CPU113.RCU
>>>      109693 ±  3%      +8.1%     118569 ±  3%  softirqs.CPU113.TIMER
>>>      134313 ±  3%     +13.9%     152993 ±  3%  softirqs.CPU114.RCU
>>>      109711 ±  3%      +8.6%     119143 ±  3%  softirqs.CPU114.TIMER
>>>      134559 ±  3%     +13.8%     153115 ±  3%  softirqs.CPU115.RCU
>>>      110226 ±  3%      +8.2%     119221 ±  3%  softirqs.CPU115.TIMER
>>>      134087 ±  3%     +14.2%     153077 ±  3%  softirqs.CPU116.RCU
>>>      110225 ±  3%      +9.0%     120184 ±  3%  softirqs.CPU116.TIMER
>>>      134271 ±  3%     +14.1%     153220 ±  3%  softirqs.CPU117.RCU
>>>      109179 ±  2%      +8.3%     118274 ±  3%  softirqs.CPU117.TIMER
>>>      134281 ±  3%     +14.1%     153230 ±  3%  softirqs.CPU118.RCU
>>>      134415 ±  3%     +14.0%     153215 ±  3%  softirqs.CPU119.RCU
>>>      109514 ±  2%      +8.3%     118643 ±  3%  softirqs.CPU119.TIMER
>>>      135560 ±  3%     +13.5%     153874 ±  3%  softirqs.CPU12.RCU
>>>      110316 ±  2%      +8.5%     119734 ±  3%  softirqs.CPU12.TIMER
>>>      134177 ±  3%     +14.6%     153815 ±  3%  softirqs.CPU120.RCU
>>>      134367 ±  3%     +15.0%     154493 ±  3%  softirqs.CPU121.RCU
>>>      133972 ±  3%     +14.3%     153173 ±  3%  softirqs.CPU122.RCU
>>>      134801 ±  4%     +13.6%     153136 ±  3%  softirqs.CPU123.RCU
>>>      134709 ±  3%     +13.9%     153440 ±  3%  softirqs.CPU124.RCU
>>>      135185 ±  3%     +13.1%     152925 ±  3%  softirqs.CPU125.RCU
>>>      134977 ±  3%     +13.8%     153608 ±  3%  softirqs.CPU126.RCU
>>>      133862 ±  4%     +14.7%     153558 ±  3%  softirqs.CPU127.RCU
>>>      134554 ±  3%     +13.9%     153237 ±  3%  softirqs.CPU128.RCU
>>>      134254 ±  3%     +14.1%     153130 ±  3%  softirqs.CPU129.RCU
>>>      135133 ±  3%     +14.0%     154103 ±  3%  softirqs.CPU13.RCU
>>>      110389 ±  2%      +8.1%     119382 ±  3%  softirqs.CPU13.TIMER
>>>      134319 ±  3%     +13.8%     152909 ±  3%  softirqs.CPU130.RCU
>>>      133590 ±  3%     +14.5%     152925 ±  3%  softirqs.CPU131.RCU
>>>      134416 ±  4%     +14.0%     153223 ±  3%  softirqs.CPU132.RCU
>>>      133931 ±  3%     +14.5%     153359 ±  3%  softirqs.CPU133.RCU
>>>      134031 ±  3%     +14.5%     153527 ±  3%  softirqs.CPU134.RCU
>>>      133851 ±  4%     +14.3%     153052 ±  3%  softirqs.CPU135.RCU
>>>      134235 ±  4%     +13.9%     152959 ±  3%  softirqs.CPU136.RCU
>>>      134003 ±  3%     +14.2%     153037 ±  3%  softirqs.CPU137.RCU
>>>      133453 ±  3%     +14.4%     152625 ±  3%  softirqs.CPU138.RCU
>>>      133514 ±  3%     +14.8%     153289 ±  3%  softirqs.CPU139.RCU
>>>      134952 ±  3%     +13.6%     153372 ±  3%  softirqs.CPU14.RCU
>>>      134045 ±  3%     +13.7%     152361 ±  3%  softirqs.CPU140.RCU
>>>      133662 ±  3%     +14.1%     152528 ±  3%  softirqs.CPU141.RCU
>>>      133643 ±  3%     +14.7%     153283 ±  2%  softirqs.CPU142.RCU
>>>      133313 ±  3%     +15.0%     153254 ±  3%  softirqs.CPU143.RCU
>>>      133585 ±  3%     +14.4%     152837 ±  3%  softirqs.CPU144.RCU
>>>      108590 ±  3%     +12.7%     122339 ±  5%  softirqs.CPU144.TIMER
>>>      133471 ±  3%     +14.5%     152883 ±  3%  softirqs.CPU145.RCU
>>>      108095 ±  3%     +14.5%     123726 ±  6%  softirqs.CPU145.TIMER
>>>      133588 ±  3%     +15.4%     154173 ±  3%  softirqs.CPU146.RCU
>>>      108268 ±  3%     +19.8%     129684 ± 16%  softirqs.CPU146.TIMER
>>>      134060 ±  3%     +14.3%     153270 ±  3%  softirqs.CPU147.RCU
>>>      108731 ±  3%     +13.0%     122902 ±  5%  softirqs.CPU147.TIMER
>>>      133778 ±  3%     +14.1%     152681 ±  3%  softirqs.CPU148.RCU
>>>      108208 ±  3%     +12.6%     121875 ±  5%  softirqs.CPU148.TIMER
>>>      133514 ±  3%     +14.2%     152460 ±  3%  softirqs.CPU149.RCU
>>>      108054 ±  3%     +15.6%     124938 ±  9%  softirqs.CPU149.TIMER
>>>      134677 ±  3%     +14.5%     154138 ±  2%  softirqs.CPU15.RCU
>>>      133245 ±  3%     +14.4%     152373 ±  3%  softirqs.CPU150.RCU
>>>      107788 ±  3%     +13.1%     121957 ±  5%  softirqs.CPU150.TIMER
>>>      133747 ±  3%     +14.0%     152473 ±  3%  softirqs.CPU151.RCU
>>>      108258 ±  4%     +12.4%     121662 ±  5%  softirqs.CPU151.TIMER
>>>      133819 ±  3%     +14.1%     152726 ±  3%  softirqs.CPU152.RCU
>>>      108163 ±  3%     +12.7%     121940 ±  5%  softirqs.CPU152.TIMER
>>>      134057 ±  3%     +14.0%     152844 ±  3%  softirqs.CPU153.RCU
>>>      107680 ±  3%     +13.1%     121787 ±  5%  softirqs.CPU153.TIMER
>>>      133660 ±  3%     +14.2%     152675 ±  3%  softirqs.CPU154.RCU
>>>      107911 ±  3%     +15.4%     124485 ±  9%  softirqs.CPU154.TIMER
>>>      133349 ±  3%     +14.6%     152811 ±  3%  softirqs.CPU155.RCU
>>>      107513 ±  3%     +15.1%     123737 ±  6%  softirqs.CPU155.TIMER
>>>      133901 ±  3%     +14.0%     152615 ±  3%  softirqs.CPU156.RCU
>>>      107853 ±  3%     +13.0%     121904 ±  5%  softirqs.CPU156.TIMER
>>>      133621 ±  3%     +14.2%     152615 ±  3%  softirqs.CPU157.RCU
>>>      108040 ±  3%     +12.6%     121684 ±  5%  softirqs.CPU157.TIMER
>>>      133891 ±  3%     +14.2%     152862 ±  3%  softirqs.CPU158.RCU
>>>      108534 ±  3%     +12.7%     122300 ±  5%  softirqs.CPU158.TIMER
>>>      133925 ±  3%     +14.5%     153347 ±  3%  softirqs.CPU159.RCU
>>>      108099 ±  3%     +12.9%     122015 ±  5%  softirqs.CPU159.TIMER
>>>      134893 ±  3%     +14.5%     154389 ±  3%  softirqs.CPU16.RCU
>>>      134169 ±  3%     +13.9%     152812 ±  3%  softirqs.CPU160.RCU
>>>      108300 ±  3%     +12.9%     122318 ±  5%  softirqs.CPU160.TIMER
>>>      133818 ±  3%     +15.1%     154048 ±  4%  softirqs.CPU161.RCU
>>>      107919 ±  3%     +13.6%     122606 ±  6%  softirqs.CPU161.TIMER
>>>      133867 ±  3%     +14.2%     152856 ±  3%  softirqs.CPU162.RCU
>>>      107991 ±  3%     +13.2%     122209 ±  5%  softirqs.CPU162.TIMER
>>>      133579 ±  3%     +14.3%     152654 ±  3%  softirqs.CPU163.RCU
>>>      108317 ±  3%     +12.9%     122271 ±  5%  softirqs.CPU163.TIMER
>>>      133914 ±  3%     +14.4%     153158 ±  3%  softirqs.CPU164.RCU
>>>      133684 ±  3%     +14.4%     152895 ±  3%  softirqs.CPU165.RCU
>>>      107435 ±  3%     +12.5%     120877 ±  5%  softirqs.CPU165.TIMER
>>>      133983 ±  3%     +13.9%     152582 ±  3%  softirqs.CPU166.RCU
>>>      107432 ±  3%     +13.2%     121641 ±  5%  softirqs.CPU166.TIMER
>>>      133947 ±  3%     +13.6%     152137 ±  3%  softirqs.CPU167.RCU
>>>      134005 ±  3%     +14.5%     153385 ±  4%  softirqs.CPU168.RCU
>>>      115752 ±  4%     +12.0%     129634 ±  4%  softirqs.CPU168.TIMER
>>>      133557 ±  3%     +15.2%     153865 ±  4%  softirqs.CPU169.RCU
>>>      115884 ±  3%     +11.7%     129481 ±  4%  softirqs.CPU169.TIMER
>>>      135348 ±  3%     +13.8%     153977 ±  3%  softirqs.CPU17.RCU
>>>      110589 ±  3%      +7.9%     119298 ±  3%  softirqs.CPU17.TIMER
>>>      134463 ±  3%     +14.0%     153229 ±  3%  softirqs.CPU170.RCU
>>>      133740 ±  3%     +14.1%     152630 ±  3%  softirqs.CPU171.RCU
>>>      116524 ±  4%     +10.9%     129237 ±  3%  softirqs.CPU171.TIMER
>>>      133716 ±  3%     +13.8%     152196 ±  3%  softirqs.CPU172.RCU
>>>      116393 ±  4%     +10.9%     129087 ±  4%  softirqs.CPU172.TIMER
>>>      133564 ±  3%     +13.9%     152165 ±  3%  softirqs.CPU173.RCU
>>>      116237 ±  3%��    +11.4%     129460 ±  3%  softirqs.CPU173.TIMER
>>>      133411 ±  3%     +14.5%     152764 ±  3%  softirqs.CPU174.RCU
>>>      116018 ±  3%     +11.2%     129006 ±  4%  softirqs.CPU174.TIMER
>>>      133147 ±  3%     +15.0%     153104 ±  2%  softirqs.CPU175.RCU
>>>      116052 ±  4%     +11.7%     129664 ±  3%  softirqs.CPU175.TIMER
>>>      133989 ±  3%     +13.8%     152504 ±  3%  softirqs.CPU176.RCU
>>>      116071 ±  4%     +10.9%     128756 ±  3%  softirqs.CPU176.TIMER
>>>      133902 ±  3%     +14.1%     152811 ±  3%  softirqs.CPU177.RCU
>>>      115994 ±  3%     +11.1%     128849 ±  4%  softirqs.CPU177.TIMER
>>>      133679 ±  3%     +14.4%     152887 ±  3%  softirqs.CPU178.RCU
>>>      115942 ±  4%     +13.4%     131439 ±  5%  softirqs.CPU178.TIMER
>>>      133985 ±  3%     +14.0%     152762 ±  3%  softirqs.CPU179.RCU
>>>      116347 ±  4%     +11.2%     129422 ±  4%  softirqs.CPU179.TIMER
>>>      134982 ±  3%     +14.0%     153838 ±  3%  softirqs.CPU18.RCU
>>>      110310 ±  3%      +8.6%     119851 ±  3%  softirqs.CPU18.TIMER
>>>      133236 ±  3%     +14.7%     152870 ±  3%  softirqs.CPU180.RCU
>>>      115901 ±  4%     +11.3%     129054 ±  4%  softirqs.CPU180.TIMER
>>>      133615 ±  3%     +14.4%     152876 ±  3%  softirqs.CPU181.RCU
>>>      115505 ±  4%     +11.3%     128560 ±  3%  softirqs.CPU181.TIMER
>>>      133171 ±  3%     +14.7%     152810 ±  3%  softirqs.CPU182.RCU
>>>      115955 ±  4%     +11.0%     128687 ±  3%  softirqs.CPU182.TIMER
>>>      133413 ±  3%     +14.4%     152671 ±  3%  softirqs.CPU183.RCU
>>>      116313 ±  4%     +11.0%     129140 ±  3%  softirqs.CPU183.TIMER
>>>      133598 ±  3%     +14.8%     153388 ±  3%  softirqs.CPU184.RCU
>>>      115462 ±  3%     +13.0%     130430 ±  6%  softirqs.CPU184.TIMER
>>>      133884 ±  3%     +13.9%     152525 ±  3%  softirqs.CPU185.RCU
>>>      115598 ±  4%     +11.1%     128397 ±  4%  softirqs.CPU185.TIMER
>>>      133262 ±  3%     +14.5%     152646 ±  3%  softirqs.CPU186.RCU
>>>      115745 ±  3%     +11.1%     128612 ±  4%  softirqs.CPU186.TIMER
>>>      133430 ±  3%     +14.5%     152767 ±  3%  softirqs.CPU187.RCU
>>>      115769 ±  4%     +11.0%     128464 ±  3%  softirqs.CPU187.TIMER
>>>      133259 ±  3%     +14.8%     153036 ±  3%  softirqs.CPU188.RCU
>>>      116108 ±  4%     +11.3%     129232 ±  3%  softirqs.CPU188.TIMER
>>>      133609 ±  3%     +14.1%     152436 ±  3%  softirqs.CPU189.RCU
>>>      115984 ±  4%     +11.0%     128710 ±  3%  softirqs.CPU189.TIMER
>>>      135090 ±  3%     +13.9%     153802 ±  3%  softirqs.CPU19.RCU
>>>      110865 ±  2%      +8.2%     119936 ±  3%  softirqs.CPU19.TIMER
>>>      133685 ±  3%     +14.2%     152631 ±  3%  softirqs.CPU190.RCU
>>>      115642 ±  3%     +11.2%     128581 ±  4%  softirqs.CPU190.TIMER
>>>      131031 ±  3%     +14.6%     150126 ±  3%  softirqs.CPU191.RCU
>>>      114275 ±  4%     +11.4%     127249 ±  4%  softirqs.CPU191.TIMER
>>>      134928 ±  3%     +14.3%     154165 ±  3%  softirqs.CPU2.RCU
>>>      135126 ±  3%     +14.0%     154083 ±  3%  softirqs.CPU20.RCU
>>>      111013 ±  3%      +8.6%     120549 ±  3%  softirqs.CPU20.TIMER
>>>      134663 ±  3%     +14.5%     154251 ±  3%  softirqs.CPU21.RCU
>>>      109671 ±  2%      +8.2%     118708 ±  3%  softirqs.CPU21.TIMER
>>>      135047 ±  3%     +13.9%     153766 ±  3%  softirqs.CPU22.RCU
>>>      134437 ±  3%     +14.2%     153492 ±  3%  softirqs.CPU23.RCU
>>>      109675 ±  2%      +8.3%     118824 ±  3%  softirqs.CPU23.TIMER
>>>      136114 ±  3%     +13.3%     154277 ±  3%  softirqs.CPU24.RCU
>>>      134825 ±  3%     +14.2%     154007 ±  3%  softirqs.CPU25.RCU
>>>      139234 ±  4%     +13.8%     158515 ±  3%  softirqs.CPU26.RCU
>>>      135056 ±  3%     +14.1%     154062 ±  3%  softirqs.CPU27.RCU
>>>      135307 ±  3%     +13.7%     153897 ±  3%  softirqs.CPU28.RCU
>>>      134686 ±  3%     +14.7%     154497 ±  3%  softirqs.CPU29.RCU
>>>      134688 ±  3%     +14.4%     154115 ±  2%  softirqs.CPU3.RCU
>>>      134642 ±  3%     +14.5%     154226 ±  3%  softirqs.CPU30.RCU
>>>      134773 ±  3%     +14.2%     153887 ±  3%  softirqs.CPU31.RCU
>>>      134523 ±  3%     +14.3%     153733 ±  3%  softirqs.CPU32.RCU
>>>      134488 ±  3%     +14.4%     153801 ±  3%  softirqs.CPU33.RCU
>>>      134483 ±  3%     +14.1%     153380 ±  3%  softirqs.CPU34.RCU
>>>      134886 ±  3%     +13.6%     153180 ±  3%  softirqs.CPU35.RCU
>>>      135021 ±  3%     +13.6%     153340 ±  3%  softirqs.CPU36.RCU
>>>      134064 ±  3%     +15.1%     154277 ±  3%  softirqs.CPU37.RCU
>>>      134166 ±  3%     +14.8%     153967 ±  3%  softirqs.CPU38.RCU
>>>      134488 ±  3%     +14.4%     153871 ±  3%  softirqs.CPU39.RCU
>>>      135294 ±  2%     +14.6%     155025 ±  3%  softirqs.CPU4.RCU
>>>      134598 ±  4%     +14.3%     153848 ±  3%  softirqs.CPU40.RCU
>>>      134255 ±  3%     +14.8%     154099 ±  3%  softirqs.CPU41.RCU
>>>      134334 ±  3%     +14.3%     153491 ±  3%  softirqs.CPU42.RCU
>>>      134201 ±  3%     +14.5%     153654 ±  3%  softirqs.CPU43.RCU
>>>      134260 ±  3%     +14.2%     153359 ±  3%  softirqs.CPU44.RCU
>>>      134005 ±  3%     +14.4%     153367 ±  3%  softirqs.CPU45.RCU
>>>      134236 ±  3%     +14.3%     153419 ±  3%  softirqs.CPU46.RCU
>>>      134552 ±  3%     +13.8%     153056 ±  3%  softirqs.CPU47.RCU
>>>      134623 ±  3%     +14.4%     154028 ±  3%  softirqs.CPU48.RCU
>>>      134348 ±  3%     +14.7%     154080 ±  3%  softirqs.CPU49.RCU
>>>      109020 ±  3%     +18.6%     129279 ± 14%  softirqs.CPU49.TIMER
>>>      134334 ±  3%     +15.0%     154455 ±  3%  softirqs.CPU5.RCU
>>>      110260 ±  2%      +8.8%     119951 ±  4%  softirqs.CPU5.TIMER
>>>      134377 ±  3%     +13.8%     152954 ±  3%  softirqs.CPU50.RCU
>>>      109169 ±  3%     +15.0%     125548 ±  9%  softirqs.CPU50.TIMER
>>>      134780 ±  3%     +14.2%     153967 ±  3%  softirqs.CPU51.RCU
>>>      109951 ±  3%     +12.5%     123641 ±  5%  softirqs.CPU51.TIMER
>>>      133929 ±  3%     +15.0%     154012 ±  3%  softirqs.CPU52.RCU
>>>      108665 ±  3%     +12.8%     122613 ±  5%  softirqs.CPU52.TIMER
>>>      133761 ±  3%     +14.9%     153675 ±  3%  softirqs.CPU53.RCU
>>>      108606 ±  3%     +19.6%     129861 ± 16%  softirqs.CPU53.TIMER
>>>      134369 ±  3%     +14.1%     153255 ±  3%  softirqs.CPU54.RCU
>>>      108917 ±  3%     +12.5%     122585 ±  5%  softirqs.CPU54.TIMER
>>>      133810 ±  3%     +14.6%     153282 ±  3%  softirqs.CPU55.RCU
>>>      108875 ±  3%     +12.5%     122468 ±  5%  softirqs.CPU55.TIMER
>>>      134038 ±  3%     +14.3%     153248 ±  3%  softirqs.CPU56.RCU
>>>      108569 ±  3%     +12.8%     122502 ±  5%  softirqs.CPU56.TIMER
>>>      133993 ±  4%     +14.5%     153426 ±  3%  softirqs.CPU57.RCU
>>>      108465 ±  3%     +12.8%     122399 ±  5%  softirqs.CPU57.TIMER
>>>      134049 ±  3%     +14.7%     153708 ±  3%  softirqs.CPU58.RCU
>>>      108634 ±  4%     +18.8%     129107 ± 16%  softirqs.CPU58.TIMER
>>>      134025 ±  3%     +14.9%     154015 ±  3%  softirqs.CPU59.RCU
>>>      108501 ±  3%     +18.6%     128721 ± 13%  softirqs.CPU59.TIMER
>>>      135247 ±  3%     +14.5%     154804 ±  3%  softirqs.CPU6.RCU
>>>      133745 ±  4%     +14.7%     153452 ±  3%  softirqs.CPU60.RCU
>>>      108526 ±  3%     +13.0%     122688 ±  5%  softirqs.CPU60.TIMER
>>>      133931 ±  3%     +14.6%     153549 ±  3%  softirqs.CPU61.RCU
>>>      108764 ±  3%     +12.9%     122787 ±  5%  softirqs.CPU61.TIMER
>>>      134682 ±  3%     +13.8%     153266 ±  3%  softirqs.CPU62.RCU
>>>      109188 ±  3%     +12.5%     122806 ±  5%  softirqs.CPU62.TIMER
>>>      134190 ±  3%     +14.6%     153761 ±  2%  softirqs.CPU63.RCU
>>>      108489 ±  3%     +12.8%     122426 ±  5%  softirqs.CPU63.TIMER
>>>      134280 ±  3%     +14.3%     153538 ±  3%  softirqs.CPU64.RCU
>>>      108493 ±  3%     +13.3%     122953 ±  5%  softirqs.CPU64.TIMER
>>>      134074 ±  3%     +14.7%     153765 ±  3%  softirqs.CPU65.RCU
>>>      108331 ±  3%     +13.0%     122440 ±  5%  softirqs.CPU65.TIMER
>>>      134424 ±  4%     +14.3%     153692 ±  3%  softirqs.CPU66.RCU
>>>      108874 ±  3%     +12.5%     122490 ±  5%  softirqs.CPU66.TIMER
>>>      133801 ±  3%     +14.9%     153746 ±  3%  softirqs.CPU67.RCU
>>>      108903 ±  3%     +13.2%     123240 ±  5%  softirqs.CPU67.TIMER
>>>      134085 ±  3%     +14.4%     153371 ±  3%  softirqs.CPU68.RCU
>>>      109310 ±  3%     +12.3%     122810 ±  5%  softirqs.CPU68.TIMER
>>>      134091 ±  3%     +14.5%     153515 ±  3%  softirqs.CPU69.RCU
>>>      108193 ±  3%     +12.5%     121716 ±  5%  softirqs.CPU69.TIMER
>>>      134883 ±  3%     +13.6%     153181 ±  3%  softirqs.CPU7.RCU
>>>      134392 ±  3%     +14.8%     154240 ±  3%  softirqs.CPU70.RCU
>>>      108056 ±  3%     +13.4%     122578 ±  5%  softirqs.CPU70.TIMER
>>>      133901 ±  3%     +14.2%     152873 ±  3%  softirqs.CPU71.RCU
>>>      134273 ±  3%     +14.4%     153630 ±  3%  softirqs.CPU72.RCU
>>>      116189 ±  3%     +11.5%     129495 ±  3%  softirqs.CPU72.TIMER
>>>      134231 ±  3%     +14.5%     153682 ±  3%  softirqs.CPU73.RCU
>>>      116348 ±  3%     +11.3%     129453 ±  3%  softirqs.CPU73.TIMER
>>>      134235 ±  3%     +14.1%     153158 ±  3%  softirqs.CPU74.RCU
>>>      133896 ±  3%     +14.6%     153477 ±  3%  softirqs.CPU75.RCU
>>>      116933 ±  4%     +11.2%     129980 ±  3%  softirqs.CPU75.TIMER
>>>      134014 ±  3%     +14.3%     153198 ±  3%  softirqs.CPU76.RCU
>>>      116889 ±  3%     +11.0%     129768 ±  3%  softirqs.CPU76.TIMER
>>>      134608 ±  3%     +13.7%     152994 ±  3%  softirqs.CPU77.RCU
>>>      117342 ±  3%     +11.0%     130274 ±  3%  softirqs.CPU77.TIMER
>>>      134551 ±  3%     +13.9%     153253 ±  3%  softirqs.CPU78.RCU
>>>      117229 ±  4%     +10.5%     129549 ±  4%  softirqs.CPU78.TIMER
>>>      134118 ±  3%     +14.3%     153325 ±  3%  softirqs.CPU79.RCU
>>>      116982 ±  3%     +11.0%     129802 ±  3%  softirqs.CPU79.TIMER
>>>      135303 ±  3%     +13.9%     154129 ±  3%  softirqs.CPU8.RCU
>>>      134699 ±  3%     +13.6%     153047 ±  3%  softirqs.CPU80.RCU
>>>      117058 ±  4%     +10.6%     129470 ±  3%  softirqs.CPU80.TIMER
>>>      134088 ±  3%     +15.6%     155063 ±  3%  softirqs.CPU81.RCU
>>>      116539 ±  4%     +12.3%     130886 ±  5%  softirqs.CPU81.TIMER
>>>      133989 ±  3%     +14.6%     153526 ±  3%  softirqs.CPU82.RCU
>>>      116614 ±  4%     +16.8%     136238 ± 12%  softirqs.CPU82.TIMER
>>>      133966 ±  3%     +14.3%     153142 ±  3%  softirqs.CPU83.RCU
>>>      117140 ±  3%     +11.2%     130205 ±  4%  softirqs.CPU83.TIMER
>>>      133561 ±  3%     +15.0%     153661 ±  3%  softirqs.CPU84.RCU
>>>      116648 ±  4%     +11.3%     129844 ±  4%  softirqs.CPU84.TIMER
>>>      134217 ±  3%     +14.2%     153330 ±  3%  softirqs.CPU85.RCU
>>>      116247 ±  4%     +11.3%     129336 ±  3%  softirqs.CPU85.TIMER
>>>      133956 ±  3%     +14.5%     153338 ±  3%  softirqs.CPU86.RCU
>>>      116654 ±  4%     +11.1%     129560 ±  3%  softirqs.CPU86.TIMER
>>>      133834 ±  3%     +15.1%     154049 ±  2%  softirqs.CPU87.RCU
>>>      116898 ±  4%     +12.0%     130912 ±  3%  softirqs.CPU87.TIMER
>>>      133951 ±  3%     +15.1%     154211 ±  3%  softirqs.CPU88.RCU
>>>      116139 ±  3%     +16.0%     134676 ± 13%  softirqs.CPU88.TIMER
>>>      134288 ±  3%     +14.3%     153433 ±  2%  softirqs.CPU89.RCU
>>>      116227 ±  3%     +11.0%     129064 ±  3%  softirqs.CPU89.TIMER
>>>      135289 ±  3%     +13.6%     153744 ±  3%  softirqs.CPU9.RCU
>>>      133598 ±  3%     +14.5%     153019 ±  3%  softirqs.CPU90.RCU
>>>      116395 ±  3%     +10.9%     129084 ±  4%  softirqs.CPU90.TIMER
>>>      133854 ±  4%     +14.9%     153799 ±  3%  softirqs.CPU91.RCU
>>>      116562 ±  4%     +11.1%     129452 ±  3%  softirqs.CPU91.TIMER
>>>      134107 ±  3%     +14.1%     152958 ±  3%  softirqs.CPU92.RCU
>>>      117107 ±  4%     +10.6%     129559 ±  3%  softirqs.CPU92.TIMER
>>>      133800 ±  3%     +14.7%     153465 ±  3%  softirqs.CPU93.RCU
>>>      116415 ±  3%     +11.5%     129823 ±  4%  softirqs.CPU93.TIMER
>>>      134092 ±  3%     +14.0%     152835 ±  3%  softirqs.CPU94.RCU
>>>      116063 ±  3%     +11.4%     129252 ±  4%  softirqs.CPU94.TIMER
>>>      134284 ±  3%     +14.1%     153230 ±  4%  softirqs.CPU95.RCU
>>>      117908 ±  3%     +11.0%     130904 ±  4%  softirqs.CPU95.TIMER
>>>      134264 ±  3%     +14.3%     153455 ±  3%  softirqs.CPU96.RCU
>>>      109819 ±  3%      +8.6%     119258 ±  3%  softirqs.CPU96.TIMER
>>>      135000 ±  3%     +13.7%     153529 ±  3%  softirqs.CPU97.RCU
>>>      134163 ±  3%     +14.2%     153253 ±  3%  softirqs.CPU98.RCU
>>>      109990 ±  3%      +8.1%     118896 ±  3%  softirqs.CPU98.TIMER
>>>      134966 ±  3%     +13.5%     153245 ±  3%  softirqs.CPU99.RCU
>>>    25776297 ±  3%     +14.3%   29452390 ±  3%  softirqs.RCU
>>>    21445010 ±  3%      +9.5%   23488096 ±  3%  softirqs.TIMER
>>>      614.00 ±  2%      +9.9%     674.50 ±  2% 
>>> interrupts.9:IO-APIC.9-fasteoi.acpi
>>>      611854 ±  2%     +10.1%     673945 ±  2% 
>>> interrupts.CPU0.LOC:Local_timer_interrupts
>>>      614.00 ±  2%      +9.9%     674.50 ±  2% 
>>> interrupts.CPU1.9:IO-APIC.9-fasteoi.acpi
>>>      611835 ±  2%     +10.1%     673883 ±  2% 
>>> interrupts.CPU1.LOC:Local_timer_interrupts
>>>      611859 ±  2%     +10.2%     673975 ±  2% 
>>> interrupts.CPU10.LOC:Local_timer_interrupts
>>>        7508 ±  5%     +15.5%       8675 ±  4% 
>>> interrupts.CPU10.RES:Rescheduling_interrupts
>>>      611802 ±  2%     +10.2%     673904 ±  2% 
>>> interrupts.CPU100.LOC:Local_timer_interrupts
>>>        7323 ±  5%     +16.1%       8503 ±  3% 
>>> interrupts.CPU100.RES:Rescheduling_interrupts
>>>      611804 ±  2%     +10.2%     673917 ±  2% 
>>> interrupts.CPU101.LOC:Local_timer_interrupts
>>>        7342 ±  3%     +13.8%       8352 ±  3% 
>>> interrupts.CPU101.RES:Rescheduling_interrupts
>>>      611780 ±  2%     +10.2%     673877 ±  2% 
>>> interrupts.CPU102.LOC:Local_timer_interrupts
>>>        7506 ±  5%     +12.2%       8424 ±  4% 
>>> interrupts.CPU102.RES:Rescheduling_interrupts
>>>      611734 ±  2%     +10.2%     673851 ±  2% 
>>> interrupts.CPU103.LOC:Local_timer_interrupts
>>>        7436 ±  3%     +15.8%       8607 ±  4% 
>>> interrupts.CPU103.RES:Rescheduling_interrupts
>>>      611800 ±  2%     +10.2%     673908 ±  2% 
>>> interrupts.CPU104.LOC:Local_timer_interrupts
>>>        7519 ±  5%     +13.1%       8502 ±  2% 
>>> interrupts.CPU104.RES:Rescheduling_interrupts
>>>      611790 ±  2%     +10.1%     673864 ±  2% 
>>> interrupts.CPU105.LOC:Local_timer_interrupts
>>>        7346 ±  5%     +16.4%       8553 ±  2% 
>>> interrupts.CPU105.RES:Rescheduling_interrupts
>>>      611801 ±  2%     +10.1%     673859 ±  2% 
>>> interrupts.CPU106.LOC:Local_timer_interrupts
>>>        7352 ±  5%     +16.5%       8567 ±  2% 
>>> interrupts.CPU106.RES:Rescheduling_interrupts
>>>      611797 ±  2%     +10.1%     673890 ±  2% 
>>> interrupts.CPU107.LOC:Local_timer_interrupts
>>>        7238 ±  6%     +15.3%       8344 ±  4% 
>>> interrupts.CPU107.RES:Rescheduling_interrupts
>>>      611778 ±  2%     +10.2%     673882 ±  2% 
>>> interrupts.CPU108.LOC:Local_timer_interrupts
>>>        7310 ±  5%     +16.3%       8503 ±  5% 
>>> interrupts.CPU108.RES:Rescheduling_interrupts
>>>      611788 ±  2%     +10.1%     673881 ±  2% 
>>> interrupts.CPU109.LOC:Local_timer_interrupts
>>>        7324 ±  6%     +13.4%       8307 
>>> interrupts.CPU109.RES:Rescheduling_interrupts
>>>      611827 ±  2%     +10.2%     673960 ±  2% 
>>> interrupts.CPU11.LOC:Local_timer_interrupts
>>>        7598 ±  6%     +12.6%       8557 ±  5% 
>>> interrupts.CPU11.RES:Rescheduling_interrupts
>>>      611796 ±  2%     +10.1%     673865 ±  2% 
>>> interrupts.CPU110.LOC:Local_timer_interrupts
>>>        7326 ±  4%     +16.2%       8513 ±  4% 
>>> interrupts.CPU110.RES:Rescheduling_interrupts
>>>      611804 ±  2%     +10.1%     673890 ±  2% 
>>> interrupts.CPU111.LOC:Local_timer_interrupts
>>>        7312 ±  3%     +14.4%       8365 ±  3% 
>>> interrupts.CPU111.RES:Rescheduling_interrupts
>>>      611801 ±  2%     +10.1%     673890 ±  2% 
>>> interrupts.CPU112.LOC:Local_timer_interrupts
>>>        7277 ±  2%     +14.6%       8337 ±  5% 
>>> interrupts.CPU112.RES:Rescheduling_interrupts
>>>      611789 ±  2%     +10.2%     673896 ±  2% 
>>> interrupts.CPU113.LOC:Local_timer_interrupts
>>>        7346 ±  4%     +15.2%       8461 ±  4% 
>>> interrupts.CPU113.RES:Rescheduling_interrupts
>>>      611759 ±  2%     +10.2%     673861 ±  2% 
>>> interrupts.CPU114.LOC:Local_timer_interrupts
>>>        7290 ±  4%     +17.9%       8594 ±  4% 
>>> interrupts.CPU114.RES:Rescheduling_interrupts
>>>      611783 ±  2%     +10.2%     673883 ±  2% 
>>> interrupts.CPU115.LOC:Local_timer_interrupts
>>>        7470 ±  4%     +13.6%       8489 ±  5% 
>>> interrupts.CPU115.RES:Rescheduling_interrupts
>>>      611807 ±  2%     +10.2%     673925 ±  2% 
>>> interrupts.CPU116.LOC:Local_timer_interrupts
>>>        7336 ±  4%     +15.3%       8455 ±  2% 
>>> interrupts.CPU116.RES:Rescheduling_interrupts
>>>      611778 ±  2%     +10.2%     673890 ±  2% 
>>> interrupts.CPU117.LOC:Local_timer_interrupts
>>>        7326 ±  5%     +15.9%       8493 ±  3% 
>>> interrupts.CPU117.RES:Rescheduling_interrupts
>>>      611782 ±  2%     +10.1%     673849 ±  2% 
>>> interrupts.CPU118.LOC:Local_timer_interrupts
>>>        7239 ±  4%     +18.4%       8574 ±  5% 
>>> interrupts.CPU118.RES:Rescheduling_interrupts
>>>      611778 ±  2%     +10.2%     673881 ±  2% 
>>> interrupts.CPU119.LOC:Local_timer_interrupts
>>>        7327 ±  4%     +13.7%       8335 ±  6% 
>>> interrupts.CPU119.RES:Rescheduling_interrupts
>>>      611853 ±  2%     +10.1%     673944 ±  2% 
>>> interrupts.CPU12.LOC:Local_timer_interrupts
>>>        7516 ±  4%     +17.0%       8798 ±  6% 
>>> interrupts.CPU12.RES:Rescheduling_interrupts
>>>      611417 ±  2%     +10.2%     673606 ±  2% 
>>> interrupts.CPU120.LOC:Local_timer_interrupts
>>>        7392 ±  4%     +14.2%       8444 ±  5% 
>>> interrupts.CPU120.RES:Rescheduling_interrupts
>>>      611457 ±  2%     +10.2%     673608 ±  2% 
>>> interrupts.CPU121.LOC:Local_timer_interrupts
>>>        7376 ±  3%     +15.6%       8525 ±  4% 
>>> interrupts.CPU121.RES:Rescheduling_interrupts
>>>      611426 ±  2%     +10.2%     673610 ±  2% 
>>> interrupts.CPU122.LOC:Local_timer_interrupts
>>>        7466 ±  5%     +11.7%       8336 ±  4% 
>>> interrupts.CPU122.RES:Rescheduling_interrupts
>>>      611384 ±  2%     +10.2%     673577 ±  2% 
>>> interrupts.CPU123.LOC:Local_timer_interrupts
>>>        7240 ±  5%     +18.2%       8559 ±  3% 
>>> interrupts.CPU123.RES:Rescheduling_interrupts
>>>      611388 ±  2%     +10.2%     673597 ±  2% 
>>> interrupts.CPU124.LOC:Local_timer_interrupts
>>>        7245 ±  5%     +18.0%       8548 ±  5% 
>>> interrupts.CPU124.RES:Rescheduling_interrupts
>>>      611391 ±  2%     +10.2%     673567 ±  2% 
>>> interrupts.CPU125.LOC:Local_timer_interrupts
>>>        7318 ±  3%     +15.3%       8436 ±  5% 
>>> interrupts.CPU125.RES:Rescheduling_interrupts
>>>      611421 ±  2%     +10.2%     673570 ±  2% 
>>> interrupts.CPU126.LOC:Local_timer_interrupts
>>>        7258 ±  5%     +16.4%       8448 ±  4% 
>>> interrupts.CPU126.RES:Rescheduling_interrupts
>>>      611468 ±  2%     +10.2%     673614 ±  2% 
>>> interrupts.CPU127.LOC:Local_timer_interrupts
>>>        7276 ±  5%     +16.9%       8508 ±  5% 
>>> interrupts.CPU127.RES:Rescheduling_interrupts
>>>      611422 ±  2%     +10.2%     673591 ±  2% 
>>> interrupts.CPU128.LOC:Local_timer_interrupts
>>>        7278 ±  4%     +17.0%       8518 ±  3% 
>>> interrupts.CPU128.RES:Rescheduling_interrupts
>>>      611407 ±  2%     +10.2%     673562 ±  2% 
>>> interrupts.CPU129.LOC:Local_timer_interrupts
>>>        7284 ±  3%     +17.0%       8519 ±  5% 
>>> interrupts.CPU129.RES:Rescheduling_interrupts
>>>      611825 ±  2%     +10.1%     673920 ±  2% 
>>> interrupts.CPU13.LOC:Local_timer_interrupts
>>>        7550 ±  4%     +16.2%       8771 ±  3% 
>>> interrupts.CPU13.RES:Rescheduling_interrupts
>>>      611428 ±  2%     +10.2%     673571 ±  2% 
>>> interrupts.CPU130.LOC:Local_timer_interrupts
>>>        7337 ±  5%     +16.4%       8541 ±  5% 
>>> interrupts.CPU130.RES:Rescheduling_interrupts
>>>      611406 ±  2%     +10.2%     673582 ±  2% 
>>> interrupts.CPU131.LOC:Local_timer_interrupts
>>>        7205 ±  5%     +17.1%       8440 ±  6% 
>>> interrupts.CPU131.RES:Rescheduling_interrupts
>>>      611449 ±  2%     +10.2%     673586 ±  2% 
>>> interrupts.CPU132.LOC:Local_timer_interrupts
>>>        7431 ±  4%     +12.7%       8372 ±  3% 
>>> interrupts.CPU132.RES:Rescheduling_interrupts
>>>      611464 ±  2%     +10.2%     673588 ±  2% 
>>> interrupts.CPU133.LOC:Local_timer_interrupts
>>>        7289 ±  4%     +15.9%       8446 ±  3% 
>>> interrupts.CPU133.RES:Rescheduling_interrupts
>>>      611437 ±  2%     +10.2%     673583 ±  2% 
>>> interrupts.CPU134.LOC:Local_timer_interrupts
>>>        7321 ±  4%     +17.7%       8616 ±  4% 
>>> interrupts.CPU134.RES:Rescheduling_interrupts
>>>      611401 ±  2%     +10.2%     673595 ±  2% 
>>> interrupts.CPU135.LOC:Local_timer_interrupts
>>>        7300 ±  4%     +15.7%       8446 ±  4% 
>>> interrupts.CPU135.RES:Rescheduling_interrupts
>>>      611462 ±  2%     +10.2%     673602 ±  2% 
>>> interrupts.CPU136.LOC:Local_timer_interrupts
>>>        7389 ±  4%     +13.9%       8417 ±  5% 
>>> interrupts.CPU136.RES:Rescheduling_interrupts
>>>      611424 ±  2%     +10.2%     673588 ±  2% 
>>> interrupts.CPU137.LOC:Local_timer_interrupts
>>>        7126 ±  4%     +20.0%       8550 ±  5% 
>>> interrupts.CPU137.RES:Rescheduling_interrupts
>>>      611383 ±  2%     +10.2%     673556 ±  2% 
>>> interrupts.CPU138.LOC:Local_timer_interrupts
>>>        7297 ±  5%     +16.1%       8475 ±  5% 
>>> interrupts.CPU138.RES:Rescheduling_interrupts
>>>      611422 ±  2%     +10.2%     673603 ±  2% 
>>> interrupts.CPU139.LOC:Local_timer_interrupts
>>>        7160 ±  4%     +17.6%       8423 ±  4% 
>>> interrupts.CPU139.RES:Rescheduling_interrupts
>>>      611815 ±  2%     +10.1%     673901 ±  2% 
>>> interrupts.CPU14.LOC:Local_timer_interrupts
>>>        7576 ±  5%     +16.3%       8810 ±  3% 
>>> interrupts.CPU14.RES:Rescheduling_interrupts
>>>      611388 ±  2%     +10.2%     673577 ±  2% 
>>> interrupts.CPU140.LOC:Local_timer_interrupts
>>>        7342 ±  6%     +12.9%       8293 ±  4% 
>>> interrupts.CPU140.RES:Rescheduling_interrupts
>>>      611394 ±  2%     +10.2%     673596 ±  2% 
>>> interrupts.CPU141.LOC:Local_timer_interrupts
>>>        7257 ±  4%     +16.1%       8423 ±  5% 
>>> interrupts.CPU141.RES:Rescheduling_interrupts
>>>      611411 ±  2%     +10.2%     673590 ±  2% 
>>> interrupts.CPU142.LOC:Local_timer_interrupts
>>>        7300 ±  6%     +16.1%       8475 ±  3% 
>>> interrupts.CPU142.RES:Rescheduling_interrupts
>>>      611396 ±  2%     +10.2%     673622 ±  2% 
>>> interrupts.CPU143.LOC:Local_timer_interrupts
>>>        7399 ±  5%     +14.0%       8435 ±  4% 
>>> interrupts.CPU143.RES:Rescheduling_interrupts
>>>      610929 ±  2%     +10.3%     673686 ±  2% 
>>> interrupts.CPU144.LOC:Local_timer_interrupts
>>>        7281 ±  5%     +13.4%       8257 ±  5% 
>>> interrupts.CPU144.RES:Rescheduling_interrupts
>>>      610918 ±  2%     +10.3%     673755 ±  2% 
>>> interrupts.CPU145.LOC:Local_timer_interrupts
>>>        7343 ±  5%     +14.5%       8408 ±  4% 
>>> interrupts.CPU145.RES:Rescheduling_interrupts
>>>      610910 ±  2%     +10.3%     673705 ±  2% 
>>> interrupts.CPU146.LOC:Local_timer_interrupts
>>>        7524 ±  4%     +13.6%       8550 ±  6% 
>>> interrupts.CPU146.RES:Rescheduling_interrupts
>>>      610858 ±  2%     +10.3%     673769 ±  2% 
>>> interrupts.CPU147.LOC:Local_timer_interrupts
>>>        7433 ±  4%     +10.8%       8237 ±  3% 
>>> interrupts.CPU147.RES:Rescheduling_interrupts
>>>      610872 ±  2%     +10.3%     673727 ±  2% 
>>> interrupts.CPU148.LOC:Local_timer_interrupts
>>>        7354 ±  3%     +13.1%       8320 ±  4% 
>>> interrupts.CPU148.RES:Rescheduling_interrupts
>>>      610909 ±  2%     +10.3%     673750 ±  2% 
>>> interrupts.CPU149.LOC:Local_timer_interrupts
>>>        7365 ±  3%     +16.5%       8582 ±  3% 
>>> interrupts.CPU149.RES:Rescheduling_interrupts
>>>      611818 ±  2%     +10.2%     673937 ±  2% 
>>> interrupts.CPU15.LOC:Local_timer_interrupts
>>>        7632 ±  4%     +12.5%       8587 ±  4% 
>>> interrupts.CPU15.RES:Rescheduling_interrupts
>>>      610898 ±  2%     +10.3%     673770 ±  2% 
>>> interrupts.CPU150.LOC:Local_timer_interrupts
>>>        7369 ±  4%     +12.4%       8282 ±  3% 
>>> interrupts.CPU150.RES:Rescheduling_interrupts
>>>      610887 ±  2%     +10.3%     673754 ±  2% 
>>> interrupts.CPU151.LOC:Local_timer_interrupts
>>>        7322 ±  4%     +15.3%       8445 ±  3% 
>>> interrupts.CPU151.RES:Rescheduling_interrupts
>>>      610849 ±  2%     +10.3%     673762 ±  2% 
>>> interrupts.CPU152.LOC:Local_timer_interrupts
>>>        7305 ±  4%     +15.5%       8439 ±  5% 
>>> interrupts.CPU152.RES:Rescheduling_interrupts
>>>      610909 ±  2%     +10.3%     673759 ±  2% 
>>> interrupts.CPU153.LOC:Local_timer_interrupts
>>>        7363 ±  6%     +14.5%       8432 ±  2% 
>>> interrupts.CPU153.RES:Rescheduling_interrupts
>>>      610878 ±  2%     +10.3%     673730 ±  2% 
>>> interrupts.CPU154.LOC:Local_timer_interrupts
>>>        7373 ±  4%     +14.1%       8414 ±  4% 
>>> interrupts.CPU154.RES:Rescheduling_interrupts
>>>      610891 ±  2%     +10.3%     673720 ±  2% 
>>> interrupts.CPU155.LOC:Local_timer_interrupts
>>>        7385 ±  5%     +11.8%       8260 ±  4% 
>>> interrupts.CPU155.RES:Rescheduling_interrupts
>>>      610909 ±  2%     +10.3%     673753 ±  2% 
>>> interrupts.CPU156.LOC:Local_timer_interrupts
>>>        7377 ±  5%     +14.0%       8410 ±  5% 
>>> interrupts.CPU156.RES:Rescheduling_interrupts
>>>      610886 ±  2%     +10.3%     673754 ±  2% 
>>> interrupts.CPU157.LOC:Local_timer_interrupts
>>>        7399 ±  5%     +13.1%       8370 ±  4% 
>>> interrupts.CPU157.RES:Rescheduling_interrupts
>>>      610854 ±  2%     +10.3%     673699 ±  2% 
>>> interrupts.CPU158.LOC:Local_timer_interrupts
>>>        7489 ±  4%     +10.5%       8278 ±  2% 
>>> interrupts.CPU158.RES:Rescheduling_interrupts
>>>      610921 ±  2%     +10.3%     673726 ±  2% 
>>> interrupts.CPU159.LOC:Local_timer_interrupts
>>>        7281 ±  5%     +15.4%       8402 ±  4% 
>>> interrupts.CPU159.RES:Rescheduling_interrupts
>>>      611836 ±  2%     +10.1%     673889 ±  2% 
>>> interrupts.CPU16.LOC:Local_timer_interrupts
>>>        7548 ±  4%     +15.6%       8723 ±  5% 
>>> interrupts.CPU16.RES:Rescheduling_interrupts
>>>      610870 ±  2%     +10.3%     673742 ±  2% 
>>> interrupts.CPU160.LOC:Local_timer_interrupts
>>>      610856 ±  2%     +10.3%     673722 ±  2% 
>>> interrupts.CPU161.LOC:Local_timer_interrupts
>>>        7472 ±  5%     +13.4%       8475 ±  5% 
>>> interrupts.CPU161.RES:Rescheduling_interrupts
>>>      610891 ±  2%     +10.3%     673708 ±  2% 
>>> interrupts.CPU162.LOC:Local_timer_interrupts
>>>        7177 ±  5%     +14.8%       8239 ±  5% 
>>> interrupts.CPU162.RES:Rescheduling_interrupts
>>>      610864 ±  2%     +10.3%     673696 ±  2% 
>>> interrupts.CPU163.LOC:Local_timer_interrupts
>>>        7337 ±  5%     +12.7%       8266 ±  4% 
>>> interrupts.CPU163.RES:Rescheduling_interrupts
>>>      610858 ±  2%     +10.3%     673724 ±  2% 
>>> interrupts.CPU164.LOC:Local_timer_interrupts
>>>        7269 ±  5%     +15.2%       8376 ±  4% 
>>> interrupts.CPU164.RES:Rescheduling_interrupts
>>>      610861 ±  2%     +10.3%     673711 ±  2% 
>>> interrupts.CPU165.LOC:Local_timer_interrupts
>>>        7431 ±  4%      +9.4%       8131 ±  4% 
>>> interrupts.CPU165.RES:Rescheduling_interrupts
>>>      610867 ±  2%     +10.3%     673684 ±  2% 
>>> interrupts.CPU166.LOC:Local_timer_interrupts
>>>        7339 ±  2%     +14.1%       8374 ±  4% 
>>> interrupts.CPU166.RES:Rescheduling_interrupts
>>>      610913 ±  2%     +10.3%     673730 ±  2% 
>>> interrupts.CPU167.LOC:Local_timer_interrupts
>>>        7174 ±  3%     +17.5%       8427 ±  4% 
>>> interrupts.CPU167.RES:Rescheduling_interrupts
>>>      611289 ±  2%     +10.2%     673888 ±  2% 
>>> interrupts.CPU168.LOC:Local_timer_interrupts
>>>        7311 ±  5%     +14.7%       8382 ±  3% 
>>> interrupts.CPU168.RES:Rescheduling_interrupts
>>>      611333 ±  2%     +10.2%     673887 ±  2% 
>>> interrupts.CPU169.LOC:Local_timer_interrupts
>>>        7232 ±  2%     +17.8%       8517 ±  5% 
>>> interrupts.CPU169.RES:Rescheduling_interrupts
>>>      611838 ±  2%     +10.1%     673879 ±  2% 
>>> interrupts.CPU17.LOC:Local_timer_interrupts
>>>        7539 ±  2%     +17.7%       8873 ± 10% 
>>> interrupts.CPU17.RES:Rescheduling_interrupts
>>>      611349 ±  2%     +10.2%     673889 ±  2% 
>>> interrupts.CPU170.LOC:Local_timer_interrupts
>>>        7374 ±  2%     +14.2%       8424 ±  5% 
>>> interrupts.CPU170.RES:Rescheduling_interrupts
>>>      611297 ±  2%     +10.2%     673873 ±  2% 
>>> interrupts.CPU171.LOC:Local_timer_interrupts
>>>        7438 ±  4%     +13.9%       8471 ±  5% 
>>> interrupts.CPU171.RES:Rescheduling_interrupts
>>>      611304 ±  2%     +10.2%     673845 ±  2% 
>>> interrupts.CPU172.LOC:Local_timer_interrupts
>>>        7317 ±  3%     +16.6%       8529 ±  5% 
>>> interrupts.CPU172.RES:Rescheduling_interrupts
>>>      611297 ±  2%     +10.2%     673887 ±  2% 
>>> interrupts.CPU173.LOC:Local_timer_interrupts
>>>      611309 ±  2%     +10.2%     673858 ±  2% 
>>> interrupts.CPU174.LOC:Local_timer_interrupts
>>>        7369 ±  5%     +15.8%       8531 ±  5% 
>>> interrupts.CPU174.RES:Rescheduling_interrupts
>>>      611311 ±  2%     +10.2%     673872 ±  2% 
>>> interrupts.CPU175.LOC:Local_timer_interrupts
>>>        7352 ±  4%     +15.1%       8465 ±  4% 
>>> interrupts.CPU175.RES:Rescheduling_interrupts
>>>      611309 ±  2%     +10.2%     673886 ±  2% 
>>> interrupts.CPU176.LOC:Local_timer_interrupts
>>>        7395 ±  2%     +14.0%       8434 ±  5% 
>>> interrupts.CPU176.RES:Rescheduling_interrupts
>>>      611322 ±  2%     +10.2%     673868 ±  2% 
>>> interrupts.CPU177.LOC:Local_timer_interrupts
>>>      611248 ±  2%     +10.2%     673805 ±  2% 
>>> interrupts.CPU178.LOC:Local_timer_interrupts
>>>        7355 ±  5%     +14.1%       8393 ±  4% 
>>> interrupts.CPU178.RES:Rescheduling_interrupts
>>>      611319 ±  2%     +10.2%     673906 ±  2% 
>>> interrupts.CPU179.LOC:Local_timer_interrupts
>>>        7269 ±  4%     +15.2%       8374 ±  6% 
>>> interrupts.CPU179.RES:Rescheduling_interrupts
>>>      611799 ±  2%     +10.1%     673864 ±  2% 
>>> interrupts.CPU18.LOC:Local_timer_interrupts
>>>        7585 ±  4%     +14.5%       8683 ±  4% 
>>> interrupts.CPU18.RES:Rescheduling_interrupts
>>>      611330 ±  2%     +10.2%     673869 ±  2% 
>>> interrupts.CPU180.LOC:Local_timer_interrupts
>>>        7404 ±  4%     +14.0%       8442 ±  5% 
>>> interrupts.CPU180.RES:Rescheduling_interrupts
>>>      611324 ±  2%     +10.2%     673889 ±  2% 
>>> interrupts.CPU181.LOC:Local_timer_interrupts
>>>        7435 ±  2%     +13.0%       8399 ±  3% 
>>> interrupts.CPU181.RES:Rescheduling_interrupts
>>>      611322 ±  2%     +10.2%     673874 ±  2% 
>>> interrupts.CPU182.LOC:Local_timer_interrupts
>>>        7250 ±  4%     +14.6%       8305 ±  4% 
>>> interrupts.CPU182.RES:Rescheduling_interrupts
>>>      611307 ±  2%     +10.2%     673852 ±  2% 
>>> interrupts.CPU183.LOC:Local_timer_interrupts
>>>        7320 ±  4%     +13.2%       8290 ±  5% 
>>> interrupts.CPU183.RES:Rescheduling_interrupts
>>>      611329 ±  2%     +10.2%     673877 ±  2% 
>>> interrupts.CPU184.LOC:Local_timer_interrupts
>>>        7377 ±  4%     +12.5%       8299 ±  5% 
>>> interrupts.CPU184.RES:Rescheduling_interrupts
>>>      611312 ±  2%     +10.2%     673866 ±  2% 
>>> interrupts.CPU185.LOC:Local_timer_interrupts
>>>        7333 ±  4%     +13.9%       8351 ±  5% 
>>> interrupts.CPU185.RES:Rescheduling_interrupts
>>>      611290 ±  2%     +10.2%     673868 ±  2% 
>>> interrupts.CPU186.LOC:Local_timer_interrupts
>>>        7426 ±  4%     +14.1%       8471 ±  3% 
>>> interrupts.CPU186.RES:Rescheduling_interrupts
>>>      611304 ±  2%     +10.2%     673853 ±  2% 
>>> interrupts.CPU187.LOC:Local_timer_interrupts
>>>        7380 ±  3%     +12.5%       8304 ±  5% 
>>> interrupts.CPU187.RES:Rescheduling_interrupts
>>>      611307 ±  2%     +10.2%     673881 ±  2% 
>>> interrupts.CPU188.LOC:Local_timer_interrupts
>>>        7277 ±  5%     +16.3%       8467 ±  6% 
>>> interrupts.CPU188.RES:Rescheduling_interrupts
>>>      611299 ±  2%     +10.2%     673879 ±  2% 
>>> interrupts.CPU189.LOC:Local_timer_interrupts
>>>        7436 ±  3%     +14.5%       8514 ±  6% 
>>> interrupts.CPU189.RES:Rescheduling_interrupts
>>>      611761 ±  2%     +10.2%     673928 ±  2% 
>>> interrupts.CPU19.LOC:Local_timer_interrupts
>>>        7515 ±  6%     +12.7%       8467 ±  4% 
>>> interrupts.CPU19.RES:Rescheduling_interrupts
>>>      611321 ±  2%     +10.2%     673856 ±  2% 
>>> interrupts.CPU190.LOC:Local_timer_interrupts
>>>        7188 ±  3%     +16.4%       8366 ±  5% 
>>> interrupts.CPU190.RES:Rescheduling_interrupts
>>>      611544 ±  2%     +10.2%     674120 ±  2% 
>>> interrupts.CPU191.LOC:Local_timer_interrupts
>>>        7207 ±  3%     +15.3%       8312 ±  3% 
>>> interrupts.CPU191.RES:Rescheduling_interrupts
>>>      611805 ±  2%     +10.2%     673920 ±  2% 
>>> interrupts.CPU2.LOC:Local_timer_interrupts
>>>      611781 ±  2%     +10.2%     673897 ±  2% 
>>> interrupts.CPU20.LOC:Local_timer_interrupts
>>>        7531 ±  5%     +16.4%       8768 ±  3% 
>>> interrupts.CPU20.RES:Rescheduling_interrupts
>>>      611715 ±  2%     +10.2%     673897 ±  2% 
>>> interrupts.CPU21.LOC:Local_timer_interrupts
>>>        7467 ±  3%     +14.9%       8577 ±  4% 
>>> interrupts.CPU21.RES:Rescheduling_interrupts
>>>      611834 ±  2%     +10.1%     673900 ±  2% 
>>> interrupts.CPU22.LOC:Local_timer_interrupts
>>>        7427 ±  6%     +17.9%       8754 ±  4% 
>>> interrupts.CPU22.RES:Rescheduling_interrupts
>>>      611788 ±  2%     +10.1%     673869 ±  2% 
>>> interrupts.CPU23.LOC:Local_timer_interrupts
>>>        7592 ±  4%     +12.2%       8518 ±  4% 
>>> interrupts.CPU23.RES:Rescheduling_interrupts
>>>      611426 ±  2%     +10.2%     673619 ±  2% 
>>> interrupts.CPU24.LOC:Local_timer_interrupts
>>>        7457 ±  4%     +14.7%       8551 ±  3% 
>>> interrupts.CPU24.RES:Rescheduling_interrupts
>>>      611439 ±  2%     +10.2%     673636 ±  2% 
>>> interrupts.CPU25.LOC:Local_timer_interrupts
>>>      611402 ±  2%     +10.2%     673620 ±  2% 
>>> interrupts.CPU26.LOC:Local_timer_interrupts
>>>        7463 ±  3%     +18.4%       8838 ±  7% 
>>> interrupts.CPU26.RES:Rescheduling_interrupts
>>>      611413 ±  2%     +10.2%     673615 ±  2% 
>>> interrupts.CPU27.LOC:Local_timer_interrupts
>>>        7412 ±  4%     +16.1%       8607 ±  4% 
>>> interrupts.CPU27.RES:Rescheduling_interrupts
>>>      611409 ±  2%     +10.2%     673625 ±  2% 
>>> interrupts.CPU28.LOC:Local_timer_interrupts
>>>        7478 ±  5%     +15.6%       8648 ±  3% 
>>> interrupts.CPU28.RES:Rescheduling_interrupts
>>>      611415 ±  2%     +10.2%     673620 ±  2% 
>>> interrupts.CPU29.LOC:Local_timer_interrupts
>>>        7431 ±  2%     +15.5%       8585 ±  2% 
>>> interrupts.CPU29.RES:Rescheduling_interrupts
>>>      611807 ±  2%     +10.1%     673896 ±  2% 
>>> interrupts.CPU3.LOC:Local_timer_interrupts
>>>        7659 ±  5%     +20.9%       9264 ± 10% 
>>> interrupts.CPU3.RES:Rescheduling_interrupts
>>>      611433 ±  2%     +10.2%     673520 ±  2% 
>>> interrupts.CPU30.LOC:Local_timer_interrupts
>>>        7432 ±  3%     +16.5%       8659 ±  4% 
>>> interrupts.CPU30.RES:Rescheduling_interrupts
>>>      611450 ±  2%     +10.2%     673604 ±  2% 
>>> interrupts.CPU31.LOC:Local_timer_interrupts
>>>        7490 ±  4%     +14.6%       8585 ±  5% 
>>> interrupts.CPU31.RES:Rescheduling_interrupts
>>>      611444 ±  2%     +10.2%     673588 ±  2% 
>>> interrupts.CPU32.LOC:Local_timer_interrupts
>>>        7393 ±  4%     +19.0%       8799 ±  3% 
>>> interrupts.CPU32.RES:Rescheduling_interrupts
>>>      611407 ±  2%     +10.2%     673606 ±  2% 
>>> interrupts.CPU33.LOC:Local_timer_interrupts
>>>        7398 ±  4%     +16.8%       8644 ±  4% 
>>> interrupts.CPU33.RES:Rescheduling_interrupts
>>>      611452 ±  2%     +10.2%     673609 ±  2% 
>>> interrupts.CPU34.LOC:Local_timer_interrupts
>>>        7414 ±  5%     +18.8%       8806 ±  4% 
>>> interrupts.CPU34.RES:Rescheduling_interrupts
>>>      611433 ±  2%     +10.2%     673588 ±  2% 
>>> interrupts.CPU35.LOC:Local_timer_interrupts
>>>        7486 ±  5%     +18.6%       8878 ±  3% 
>>> interrupts.CPU35.RES:Rescheduling_interrupts
>>>      611450 ±  2%     +10.2%     673603 ±  2% 
>>> interrupts.CPU36.LOC:Local_timer_interrupts
>>>        7503 ±  3%     +15.0%       8627 ±  3% 
>>> interrupts.CPU36.RES:Rescheduling_interrupts
>>>      611465 ±  2%     +10.2%     673611 ±  2% 
>>> interrupts.CPU37.LOC:Local_timer_interrupts
>>>        7416 ±  3%     +17.6%       8719 ±  2% 
>>> interrupts.CPU37.RES:Rescheduling_interrupts
>>>      611445 ±  2%     +10.2%     673575 ±  2% 
>>> interrupts.CPU38.LOC:Local_timer_interrupts
>>>        7477 ±  5%     +14.2%       8540 ±  3% 
>>> interrupts.CPU38.RES:Rescheduling_interrupts
>>>      611420 ±  2%     +10.2%     673600 ±  2% 
>>> interrupts.CPU39.LOC:Local_timer_interrupts
>>>        7562 ±  5%     +14.0%       8620 ±  4% 
>>> interrupts.CPU39.RES:Rescheduling_interrupts
>>>      611823 ±  2%     +10.2%     673949 ±  2% 
>>> interrupts.CPU4.LOC:Local_timer_interrupts
>>>        7579 ±  4%     +14.9%       8708 ±  3% 
>>> interrupts.CPU4.RES:Rescheduling_interrupts
>>>      611390 ±  2%     +10.2%     673618 ±  2% 
>>> interrupts.CPU40.LOC:Local_timer_interrupts
>>>        7444 ±  4%     +15.9%       8626 ±  3% 
>>> interrupts.CPU40.RES:Rescheduling_interrupts
>>>      611429 ±  2%     +10.2%     673612 ±  2% 
>>> interrupts.CPU41.LOC:Local_timer_interrupts
>>>        7464 ±  5%     +15.3%       8605 ±  2% 
>>> interrupts.CPU41.RES:Rescheduling_interrupts
>>>      611418 ±  2%     +10.2%     673597 ±  2% 
>>> interrupts.CPU42.LOC:Local_timer_interrupts
>>>        7394 ±  4%     +18.6%       8772 ±  5% 
>>> interrupts.CPU42.RES:Rescheduling_interrupts
>>>      611364 ±  2%     +10.2%     673601 ±  2% 
>>> interrupts.CPU43.LOC:Local_timer_interrupts
>>>        7483 ±  5%     +15.3%       8630 ±  3% 
>>> interrupts.CPU43.RES:Rescheduling_interrupts
>>>      611421 ±  2%     +10.2%     673593 ±  2% 
>>> interrupts.CPU44.LOC:Local_timer_interrupts
>>>        7463 ±  2%     +16.0%       8656 ±  4% 
>>> interrupts.CPU44.RES:Rescheduling_interrupts
>>>      611423 ±  2%     +10.2%     673593 ±  2% 
>>> interrupts.CPU45.LOC:Local_timer_interrupts
>>>        7455 ±  4%     +14.8%       8558 ±  3% 
>>> interrupts.CPU45.RES:Rescheduling_interrupts
>>>      611333 ±  2%     +10.2%     673616 ±  2% 
>>> interrupts.CPU46.LOC:Local_timer_interrupts
>>>        7406 ±  3%     +17.4%       8696 ±  3% 
>>> interrupts.CPU46.RES:Rescheduling_interrupts
>>>      611421 ±  2%     +10.2%     673605 ±  2% 
>>> interrupts.CPU47.LOC:Local_timer_interrupts
>>>        7482 ±  5%     +15.7%       8660 ±  2% 
>>> interrupts.CPU47.RES:Rescheduling_interrupts
>>>      610910 ±  2%     +10.3%     673737 ±  2% 
>>> interrupts.CPU48.LOC:Local_timer_interrupts
>>>        7521 ±  6%     +15.1%       8655 ±  4% 
>>> interrupts.CPU48.RES:Rescheduling_interrupts
>>>      610921 ±  2%     +10.3%     673770 ±  2% 
>>> interrupts.CPU49.LOC:Local_timer_interrupts
>>>        7594 ±  5%     +12.7%       8558 ±  3% 
>>> interrupts.CPU49.RES:Rescheduling_interrupts
>>>      611796 ±  2%     +10.1%     673831 ±  2% 
>>> interrupts.CPU5.LOC:Local_timer_interrupts
>>>      610930 ±  2%     +10.3%     673671 ±  2% 
>>> interrupts.CPU50.LOC:Local_timer_interrupts
>>>        7504 ±  2%     +17.6%       8824 ±  4% 
>>> interrupts.CPU50.RES:Rescheduling_interrupts
>>>      610882 ±  2%     +10.3%     673738 ±  2% 
>>> interrupts.CPU51.LOC:Local_timer_interrupts
>>>      610884 ±  2%     +10.3%     673731 ±  2% 
>>> interrupts.CPU52.LOC:Local_timer_interrupts
>>>        7436 ±  4%     +18.1%       8783 ±  3% 
>>> interrupts.CPU52.RES:Rescheduling_interrupts
>>>      610885 ±  2%     +10.3%     673774 ±  2% 
>>> interrupts.CPU53.LOC:Local_timer_interrupts
>>>        7544 ±  4%     +15.3%       8700 ±  4% 
>>> interrupts.CPU53.RES:Rescheduling_interrupts
>>>      610902 ±  2%     +10.3%     673763 ±  2% 
>>> interrupts.CPU54.LOC:Local_timer_interrupts
>>>        7534 ±  4%     +15.6%       8709 ±  6% 
>>> interrupts.CPU54.RES:Rescheduling_interrupts
>>>      610927 ±  2%     +10.3%     673739 ±  2% 
>>> interrupts.CPU55.LOC:Local_timer_interrupts
>>>      610883 ±  2%     +10.3%     673719 ±  2% 
>>> interrupts.CPU56.LOC:Local_timer_interrupts
>>>      610921 ±  2%     +10.3%     673689 ±  2% 
>>> interrupts.CPU57.LOC:Local_timer_interrupts
>>>        7473 ±  5%     +14.8%       8577 ±  4% 
>>> interrupts.CPU57.RES:Rescheduling_interrupts
>>>      610893 ±  2%     +10.3%     673781 ±  2% 
>>> interrupts.CPU58.LOC:Local_timer_interrupts
>>>        7407 ±  5%     +16.6%       8635 ±  3% 
>>> interrupts.CPU58.RES:Rescheduling_interrupts
>>>      610898 ±  2%     +10.3%     673763 ±  2% 
>>> interrupts.CPU59.LOC:Local_timer_interrupts
>>>        7642 ±  5%     +13.1%       8643 ±  3% 
>>> interrupts.CPU59.RES:Rescheduling_interrupts
>>>      611823 ±  2%     +10.1%     673914 ±  2% 
>>> interrupts.CPU6.LOC:Local_timer_interrupts
>>>        7542 ±  3%     +17.3%       8845 ±  3% 
>>> interrupts.CPU6.RES:Rescheduling_interrupts
>>>      610869 ±  2%     +10.3%     673784 ±  2% 
>>> interrupts.CPU60.LOC:Local_timer_interrupts
>>>        7535 ±  4%     +14.8%       8647 ±  5% 
>>> interrupts.CPU60.RES:Rescheduling_interrupts
>>>      610906 ±  2%     +10.3%     673701 ±  2% 
>>> interrupts.CPU61.LOC:Local_timer_interrupts
>>>        7631 ±  4%     +13.5%       8660 ±  4% 
>>> interrupts.CPU61.RES:Rescheduling_interrupts
>>>      610863 ±  2%     +10.3%     673705 ±  2% 
>>> interrupts.CPU62.LOC:Local_timer_interrupts
>>>        7634 ±  4%     +11.8%       8532 ±  4% 
>>> interrupts.CPU62.RES:Rescheduling_interrupts
>>>      610920 ±  2%     +10.3%     673730 ±  2% 
>>> interrupts.CPU63.LOC:Local_timer_interrupts
>>>        7549 ±  5%     +14.8%       8663 ±  5% 
>>> interrupts.CPU63.RES:Rescheduling_interrupts
>>>      610884 ±  2%     +10.3%     673727 ±  2% 
>>> interrupts.CPU64.LOC:Local_timer_interrupts
>>>        7644 ±  3%     +13.4%       8672 ±  3% 
>>> interrupts.CPU64.RES:Rescheduling_interrupts
>>>      610853 ±  2%     +10.3%     673708 ±  2% 
>>> interrupts.CPU65.LOC:Local_timer_interrupts
>>>        7555 ±  5%     +14.8%       8672 ±  3% 
>>> interrupts.CPU65.RES:Rescheduling_interrupts
>>>      610950 ±  2%     +10.3%     673804 ±  2% 
>>> interrupts.CPU66.LOC:Local_timer_interrupts
>>>        7457 ±  5%     +16.8%       8710 ±  4% 
>>> interrupts.CPU66.RES:Rescheduling_interrupts
>>>      610888 ±  2%     +10.3%     673735 ±  2% 
>>> interrupts.CPU67.LOC:Local_timer_interrupts
>>>        7550 ±  4%     +11.5%       8415 ±  5% 
>>> interrupts.CPU67.RES:Rescheduling_interrupts
>>>      610891 ±  2%     +10.3%     673727 ±  2% 
>>> interrupts.CPU68.LOC:Local_timer_interrupts
>>>        7612 ±  4%     +12.1%       8534 ±  3% 
>>> interrupts.CPU68.RES:Rescheduling_interrupts
>>>      610901 ±  2%     +10.3%     673659 ±  2% 
>>> interrupts.CPU69.LOC:Local_timer_interrupts
>>>        7493 ±  4%     +14.8%       8601 ±  4% 
>>> interrupts.CPU69.RES:Rescheduling_interrupts
>>>      611808 ±  2%     +10.1%     673844 ±  2% 
>>> interrupts.CPU7.LOC:Local_timer_interrupts
>>>        7673 ±  5%     +15.0%       8821 ±  4% 
>>> interrupts.CPU7.RES:Rescheduling_interrupts
>>>      610906 ±  2%     +10.3%     673733 ±  2% 
>>> interrupts.CPU70.LOC:Local_timer_interrupts
>>>        7412 ±  3%     +16.9%       8664 ±  4% 
>>> interrupts.CPU70.RES:Rescheduling_interrupts
>>>      610871 ±  2%     +10.3%     673671 ±  2% 
>>> interrupts.CPU71.LOC:Local_timer_interrupts
>>>        7519 ±  4%     +13.8%       8557 ±  5% 
>>> interrupts.CPU71.RES:Rescheduling_interrupts
>>>      611325 ±  2%     +10.2%     673906 ±  2% 
>>> interrupts.CPU72.LOC:Local_timer_interrupts
>>>        7720 ±  4%      +9.6%       8459 ±  4% 
>>> interrupts.CPU72.RES:Rescheduling_interrupts
>>>      611328 ±  2%     +10.2%     673908 ±  2% 
>>> interrupts.CPU73.LOC:Local_timer_interrupts
>>>        7535 ±  4%     +14.6%       8634 ±  4% 
>>> interrupts.CPU73.RES:Rescheduling_interrupts
>>>      611311 ±  2%     +10.2%     673883 ±  2% 
>>> interrupts.CPU74.LOC:Local_timer_interrupts
>>>        7526 ±  5%     +14.7%       8635 ±  6% 
>>> interrupts.CPU74.RES:Rescheduling_interrupts
>>>      611315 ±  2%     +10.2%     673868 ±  2% 
>>> interrupts.CPU75.LOC:Local_timer_interrupts
>>>        7712 ±  5%      +9.8%       8469 ±  4% 
>>> interrupts.CPU75.RES:Rescheduling_interrupts
>>>      611304 ±  2%     +10.2%     673860 ±  2% 
>>> interrupts.CPU76.LOC:Local_timer_interrupts
>>>        7476 ±  5%     +14.2%       8537 ±  5% 
>>> interrupts.CPU76.RES:Rescheduling_interrupts
>>>      611322 ±  2%     +10.2%     673834 ±  2% 
>>> interrupts.CPU77.LOC:Local_timer_interrupts
>>>        7670 ±  3%     +12.5%       8630 ±  5% 
>>> interrupts.CPU77.RES:Rescheduling_interrupts
>>>      611326 ±  2%     +10.2%     673851 ±  2% 
>>> interrupts.CPU78.LOC:Local_timer_interrupts
>>>        7417 ±  3%     +15.6%       8578 ±  4% 
>>> interrupts.CPU78.RES:Rescheduling_interrupts
>>>      611328 ±  2%     +10.2%     673820 ±  2% 
>>> interrupts.CPU79.LOC:Local_timer_interrupts
>>>        7545 ±  4%     +13.9%       8590 ±  4% 
>>> interrupts.CPU79.RES:Rescheduling_interrupts
>>>      611842 ±  2%     +10.1%     673928 ±  2% 
>>> interrupts.CPU8.LOC:Local_timer_interrupts
>>>        7673 ±  3%     +14.9%       8818 ±  4% 
>>> interrupts.CPU8.RES:Rescheduling_interrupts
>>>      611366 ±  2%     +10.2%     673868 ±  2% 
>>> interrupts.CPU80.LOC:Local_timer_interrupts
>>>        7513 ±  3%     +16.1%       8723 ±  5% 
>>> interrupts.CPU80.RES:Rescheduling_interrupts
>>>      611322 ±  2%     +10.2%     673860 ±  2% 
>>> interrupts.CPU81.LOC:Local_timer_interrupts
>>>      611299 ±  2%     +10.2%     673868 ±  2% 
>>> interrupts.CPU82.LOC:Local_timer_interrupts
>>>        7523 ±  4%     +13.8%       8562 ±  4% 
>>> interrupts.CPU82.RES:Rescheduling_interrupts
>>>      611320 ±  2%     +10.2%     673727 ±  2% 
>>> interrupts.CPU83.LOC:Local_timer_interrupts
>>>        7518 ±  5%     +12.4%       8454 ±  5% 
>>> interrupts.CPU83.RES:Rescheduling_interrupts
>>>      611326 ±  2%     +10.2%     673899 ±  2% 
>>> interrupts.CPU84.LOC:Local_timer_interrupts
>>>        7452 ±  6%     +12.3%       8365 ±  5% 
>>> interrupts.CPU84.RES:Rescheduling_interrupts
>>>      611329 ±  2%     +10.2%     673885 ±  2% 
>>> interrupts.CPU85.LOC:Local_timer_interrupts
>>>        7399 ±  5%     +11.1%       8218 ±  5% 
>>> interrupts.CPU85.RES:Rescheduling_interrupts
>>>      611278 ±  2%     +10.2%     673875 ±  2% 
>>> interrupts.CPU86.LOC:Local_timer_interrupts
>>>        7376 ±  4%     +16.3%       8581 ±  3% 
>>> interrupts.CPU86.RES:Rescheduling_interrupts
>>>      611313 ±  2%     +10.2%     673874 ±  2% 
>>> interrupts.CPU87.LOC:Local_timer_interrupts
>>>        7544 ±  4%     +12.8%       8508 ±  4% 
>>> interrupts.CPU87.RES:Rescheduling_interrupts
>>>      611351 ±  2%     +10.2%     673928 ±  2% 
>>> interrupts.CPU88.LOC:Local_timer_interrupts
>>>        7418 ±  6%     +13.3%       8406 ±  4% 
>>> interrupts.CPU88.RES:Rescheduling_interrupts
>>>      611324 ±  2%     +10.2%     673795 ±  2% 
>>> interrupts.CPU89.LOC:Local_timer_interrupts
>>>        7426 ±  4%     +13.6%       8439 ±  5% 
>>> interrupts.CPU89.RES:Rescheduling_interrupts
>>>      611832 ±  2%     +10.2%     673968 ±  2% 
>>> interrupts.CPU9.LOC:Local_timer_interrupts
>>>        7538 ±  3%     +15.6%       8715 ±  4% 
>>> interrupts.CPU9.RES:Rescheduling_interrupts
>>>      611298 ±  2%     +10.2%     673878 ±  2% 
>>> interrupts.CPU90.LOC:Local_timer_interrupts
>>>        7421 ±  3%     +12.7%       8363 ±  4% 
>>> interrupts.CPU90.RES:Rescheduling_interrupts
>>>      611305 ±  2%     +10.2%     673874 ±  2% 
>>> interrupts.CPU91.LOC:Local_timer_interrupts
>>>        7377 ±  5%     +15.6%       8528 ±  5% 
>>> interrupts.CPU91.RES:Rescheduling_interrupts
>>>      611288 ±  2%     +10.2%     673876 ±  2% 
>>> interrupts.CPU92.LOC:Local_timer_interrupts
>>>        7349 ±  4%     +12.0%       8234 ±  4% 
>>> interrupts.CPU92.RES:Rescheduling_interrupts
>>>      611329 ±  2%     +10.2%     673857 ±  2% 
>>> interrupts.CPU93.LOC:Local_timer_interrupts
>>>        7438 ±  3%     +14.7%       8530 ±  4% 
>>> interrupts.CPU93.RES:Rescheduling_interrupts
>>>      611276 ±  2%     +10.2%     673866 ±  2% 
>>> interrupts.CPU94.LOC:Local_timer_interrupts
>>>        7363 ±  4%     +12.7%       8300 ±  5% 
>>> interrupts.CPU94.RES:Rescheduling_interrupts
>>>      611381 ±  2%     +10.2%     673922 ±  2% 
>>> interrupts.CPU95.LOC:Local_timer_interrupts
>>>       10549 ±  3%     +14.9%      12117 ±  4% 
>>> interrupts.CPU95.RES:Rescheduling_interrupts
>>>      611824 ±  2%     +10.1%     673847 ±  2% 
>>> interrupts.CPU96.LOC:Local_timer_interrupts
>>>        7416 ±  3%     +12.9%       8375 ±  5% 
>>> interrupts.CPU96.RES:Rescheduling_interrupts
>>>      611751 ±  2%     +10.2%     673870 ±  2% 
>>> interrupts.CPU97.LOC:Local_timer_interrupts
>>>        7437 ±  4%     +14.0%       8477 ±  4% 
>>> interrupts.CPU97.RES:Rescheduling_interrupts
>>>      611787 ±  2%     +10.1%     673877 ±  2% 
>>> interrupts.CPU98.LOC:Local_timer_interrupts
>>>        7555 ±  4%     +13.0%       8534 ±  3% 
>>> interrupts.CPU98.RES:Rescheduling_interrupts
>>>      611783 ±  2%     +10.1%     673872 ±  2% 
>>> interrupts.CPU99.LOC:Local_timer_interrupts
>>>        7452 ±  3%     +12.8%       8404 ±  5% 
>>> interrupts.CPU99.RES:Rescheduling_interrupts
>>>   1.174e+08 ±  2%     +10.2%  1.294e+08 ±  2% 
>>> interrupts.LOC:Local_timer_interrupts
>>>     1433875 ±  3%     +14.6%    1643307 ±  3% 
>>> interrupts.RES:Rescheduling_interrupts
>>>
>>>
>>>                                    aim7.jobs-per-min
>>>    30500 
>>> +-------------------------------------------------------------------+
>>>          | ..+....                            .|
>>>    30000 |-+                       ...... ...                      ... |
>>>    29500 |................   ......                .. ...    |
>>>          |                +..                        ... ..       |
>>>    29000 |-+                                            ... 
>>> ...         |
>>>    28500 |-+                                               . 
>>> ...            |
>>>          | +.               |
>>>    28000 |-+              O O                |
>>>    27500 
>>> |-+                                                                 |
>>> |                                                                   |
>>>    27000 
>>> |-+                                                                 |
>>>    26500 
>>> |-+                                                                 |
>>>          | O                                 |
>>>    26000 
>>> +-------------------------------------------------------------------+
>>> [*] bisect-good sample
>>> [O] bisect-bad  sample
>>>
>>>
>>>
>>> Disclaimer:
>>> Results have been estimated based on internal Intel analysis and are 
>>> provided
>>> for informational purposes only. Any difference in system hardware or 
>>> software
>>> design or configuration may affect actual performance.
>>>
>>>
>>> Thanks,
>>> Rong Chen
>>>
>>>
>>> _______________________________________________
>>> LKP mailing list -- lkp@...ts.01.org
>>> To unsubscribe send an email to lkp-leave@...ts.01.org
>>>
>>
> 

-- 
Zhengjun Xing

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ