[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-ID: <20140102061242.GB29352@localhost>
Date: Thu, 2 Jan 2014 14:12:42 +0800
From: fengguang.wu@...el.com
To: Stephane Eranian <eranian@...gle.com>
Cc: Peter Zijlstra <peterz@...radead.org>,
Ingo Molnar <mingo@...nel.org>,
LKML <linux-kernel@...r.kernel.org>, lkp@...ux.intel.com
Subject: perf-stat changes after "Use hrtimers for event multiplexing"
Greetings,
We noticed many perf-stat changes between commit 9e6302056f ("perf: Use
hrtimers for event multiplexing") and its parent commit ab573844e.
Are these expected changes?
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
152917 +842.9% 1441897 TOTAL interrupts.0:IO-APIC-edge.timer
545996 +478.0% 3155637 TOTAL interrupts.LOC
182281 +12.3% 204718 TOTAL softirqs.SCHED
1.986e+08 -96.4% 7105919 TOTAL perf-stat.node-store-misses
107241719 -99.7% 317525 TOTAL perf-stat.node-prefetch-misses
1.938e+08 -90.7% 17930426 TOTAL perf-stat.node-load-misses
2590 +247.8% 9009 TOTAL vmstat.system.in
4.549e+12 +158.3% 1.175e+13 TOTAL perf-stat.stalled-cycles-backend
6.807e+12 +149.1% 1.696e+13 TOTAL perf-stat.stalled-cycles-frontend
1.753e+08 -50.8% 86339289 TOTAL perf-stat.node-prefetches
8.326e+11 +45.0% 1.207e+12 TOTAL perf-stat.cpu-cycles
37932143 +32.2% 50146025 TOTAL perf-stat.iTLB-load-misses
4.738e+11 +30.1% 6.165e+11 TOTAL perf-stat.iTLB-loads
2.56e+11 +30.1% 3.33e+11 TOTAL perf-stat.L1-icache-loads
4.951e+11 +24.6% 6.169e+11 TOTAL perf-stat.instructions
7.85e+08 +7.5% 8.439e+08 TOTAL perf-stat.LLC-prefetch-misses
1.891e+12 +22.8% 2.322e+12 TOTAL perf-stat.ref-cycles
4.344e+08 -20.3% 3.462e+08 TOTAL perf-stat.node-loads
2.836e+11 +17.4% 3.328e+11 TOTAL perf-stat.branch-loads
9.506e+10 +24.5% 1.183e+11 TOTAL perf-stat.branch-load-misses
2.803e+11 +18.4% 3.319e+11 TOTAL perf-stat.branch-instructions
7.988e+10 +20.9% 9.658e+10 TOTAL perf-stat.bus-cycles
2.041e+09 +22.2% 2.495e+09 TOTAL perf-stat.branch-misses
229145 -17.3% 189601 TOTAL perf-stat.cpu-migrations
1.782e+11 +17.9% 2.1e+11 TOTAL perf-stat.dTLB-loads
4.702e+08 -14.8% 4.006e+08 TOTAL perf-stat.LLC-load-misses
1.418e+11 +17.4% 1.666e+11 TOTAL perf-stat.L1-dcache-loads
1.838e+09 +16.1% 2.133e+09 TOTAL perf-stat.LLC-stores
2.428e+09 +11.3% 2.702e+09 TOTAL perf-stat.LLC-loads
2.788e+11 +8.6% 3.029e+11 TOTAL perf-stat.dTLB-stores
8.66e+08 +10.8% 9.594e+08 TOTAL perf-stat.LLC-prefetches
1.117e+09 +10.5% 1.234e+09 TOTAL perf-stat.dTLB-store-misses
1.705e+09 +5.3% 1.796e+09 TOTAL perf-stat.L1-dcache-store-misses
5.671e+09 +6.1% 6.015e+09 TOTAL perf-stat.L1-dcache-load-misses
8.794e+10 +3.6% 9.109e+10 TOTAL perf-stat.L1-dcache-stores
3.46e+09 +4.6% 3.618e+09 TOTAL perf-stat.cache-references
8.696e+08 +1.8% 8.849e+08 TOTAL perf-stat.cache-misses
1613129 +2.6% 1655724 TOTAL perf-stat.context-switches
All of the changes happen in one of our test box, which has a DX58SO
baseboard and 4-core CPU. The boot dmesg and kconfig are attached.
We can test more boxes if necessary.
Below is the more detailed list of changes.
ab573844e3058ee 9e6302056f8029f438e853432 <= commits being compared
--------------- -------------------------
81917 ~ 1% +773.9% 715856 ~ 6% fat/micro/dd-write/1HDD-cfq-btrfs-100dd <= test box, case and params
71000 ~ 1% +922.6% 726041 ~18% fat/micro/dd-write/1HDD-cfq-ext4-100dd <= test box, case and params
^^^^stddev ^^^^stddev
152917 +842.9% 1441897 TOTAL interrupts.0:IO-APIC-edge.timer <= metric for this chunk of compares
~~~~~~ ~~~~~~~ ~~~~~~~ value1, percent of change, value2
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
280034 ~ 1% +466.0% 1584966 ~ 2% fat/micro/dd-write/1HDD-cfq-btrfs-100dd
265962 ~ 1% +490.6% 1570671 ~ 6% fat/micro/dd-write/1HDD-cfq-ext4-100dd
545996 +478.0% 3155637 TOTAL interrupts.LOC
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
90939 ~ 2% +13.6% 103315 ~ 3% fat/micro/dd-write/1HDD-cfq-btrfs-100dd
91342 ~ 0% +11.0% 101402 ~ 0% fat/micro/dd-write/1HDD-cfq-ext4-100dd
182281 +12.3% 204718 TOTAL softirqs.SCHED
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
93997892 ~31% -93.8% 5850557 ~37% fat/micro/dd-write/1HDD-cfq-btrfs-100dd
1.046e+08 ~ 7% -98.8% 1255361 ~138% fat/micro/dd-write/1HDD-cfq-ext4-100dd
1.986e+08 -96.4% 7105919 TOTAL perf-stat.node-store-misses
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
58318221 ~10% -100.0% 18569 ~130% fat/micro/dd-write/1HDD-cfq-btrfs-100dd
48923497 ~15% -99.4% 298955 ~193% fat/micro/dd-write/1HDD-cfq-ext4-100dd
107241719 -99.7% 317525 TOTAL perf-stat.node-prefetch-misses
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
83665740 ~ 6% -82.0% 15059253 ~144% fat/micro/dd-write/1HDD-cfq-btrfs-100dd
1.102e+08 ~ 7% -97.4% 2871173 ~41% fat/micro/dd-write/1HDD-cfq-ext4-100dd
1.938e+08 -90.7% 17930426 TOTAL perf-stat.node-load-misses
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
1529 ~ 2% +208.9% 4724 ~ 2% fat/micro/dd-write/1HDD-cfq-btrfs-100dd
1061 ~ 3% +303.9% 4285 ~ 8% fat/micro/dd-write/1HDD-cfq-ext4-100dd
2590 +247.8% 9009 TOTAL vmstat.system.in
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
2.891e+12 ~ 1% +146.6% 7.13e+12 ~ 2% fat/micro/dd-write/1HDD-cfq-btrfs-100dd
1.658e+12 ~ 3% +178.6% 4.62e+12 ~ 7% fat/micro/dd-write/1HDD-cfq-ext4-100dd
4.549e+12 +158.3% 1.175e+13 TOTAL perf-stat.stalled-cycles-backend
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
4.371e+12 ~ 1% +140.0% 1.049e+13 ~ 2% fat/micro/dd-write/1HDD-cfq-btrfs-100dd
2.436e+12 ~ 3% +165.5% 6.467e+12 ~ 7% fat/micro/dd-write/1HDD-cfq-ext4-100dd
6.807e+12 +149.1% 1.696e+13 TOTAL perf-stat.stalled-cycles-frontend
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
1.753e+08 ~ 2% -50.8% 86339289 ~ 2% fat/micro/dd-write/1HDD-cfq-ext4-100dd
1.753e+08 -50.8% 86339289 TOTAL perf-stat.node-prefetches
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
8.326e+11 ~ 6% +45.0% 1.207e+12 ~ 7% fat/micro/dd-write/1HDD-cfq-ext4-100dd
8.326e+11 +45.0% 1.207e+12 TOTAL perf-stat.cpu-cycles
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
19697591 ~ 2% +32.9% 26168712 ~ 0% fat/micro/dd-write/1HDD-cfq-btrfs-100dd
18234552 ~ 3% +31.5% 23977313 ~ 4% fat/micro/dd-write/1HDD-cfq-ext4-100dd
37932143 +32.2% 50146025 TOTAL perf-stat.iTLB-load-misses
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
4.738e+11 ~ 5% +30.1% 6.165e+11 ~ 7% fat/micro/dd-write/1HDD-cfq-ext4-100dd
4.738e+11 +30.1% 6.165e+11 TOTAL perf-stat.iTLB-loads
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
2.56e+11 ~ 5% +30.1% 3.33e+11 ~ 7% fat/micro/dd-write/1HDD-cfq-ext4-100dd
2.56e+11 +30.1% 3.33e+11 TOTAL perf-stat.L1-icache-loads
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
4.951e+11 ~10% +24.6% 6.169e+11 ~ 6% fat/micro/dd-write/1HDD-cfq-ext4-100dd
4.951e+11 +24.6% 6.169e+11 TOTAL perf-stat.instructions
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
6.504e+08 ~ 7% +16.1% 7.548e+08 ~ 4% fat/micro/dd-write/1HDD-cfq-btrfs-100dd
1.345e+08 ~10% -33.8% 89058216 ~ 4% fat/micro/dd-write/1HDD-cfq-ext4-100dd
7.85e+08 +7.5% 8.439e+08 TOTAL perf-stat.LLC-prefetch-misses
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
1.055e+12 ~ 1% +5.8% 1.116e+12 ~ 1% fat/micro/dd-write/1HDD-cfq-btrfs-100dd
8.363e+11 ~ 5% +44.2% 1.206e+12 ~ 6% fat/micro/dd-write/1HDD-cfq-ext4-100dd
1.891e+12 +22.8% 2.322e+12 TOTAL perf-stat.ref-cycles
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
2.241e+08 ~ 3% -18.6% 1.826e+08 ~ 2% fat/micro/dd-write/1HDD-cfq-btrfs-100dd
2.103e+08 ~ 1% -22.2% 1.636e+08 ~ 3% fat/micro/dd-write/1HDD-cfq-ext4-100dd
4.344e+08 -20.3% 3.462e+08 TOTAL perf-stat.node-loads
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
1.856e+11 ~ 0% +4.4% 1.938e+11 ~ 1% fat/micro/dd-write/1HDD-cfq-btrfs-100dd
9.797e+10 ~12% +41.9% 1.39e+11 ~ 9% fat/micro/dd-write/1HDD-cfq-ext4-100dd
2.836e+11 +17.4% 3.328e+11 TOTAL perf-stat.branch-loads
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
9.506e+10 ~ 6% +24.5% 1.183e+11 ~ 5% fat/micro/dd-write/1HDD-cfq-ext4-100dd
9.506e+10 +24.5% 1.183e+11 TOTAL perf-stat.branch-load-misses
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
1.824e+11 ~ 0% +4.9% 1.914e+11 ~ 0% fat/micro/dd-write/1HDD-cfq-btrfs-100dd
9.791e+10 ~ 6% +43.5% 1.405e+11 ~ 9% fat/micro/dd-write/1HDD-cfq-ext4-100dd
2.803e+11 +18.4% 3.319e+11 TOTAL perf-stat.branch-instructions
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
4.411e+10 ~ 1% +5.1% 4.636e+10 ~ 0% fat/micro/dd-write/1HDD-cfq-btrfs-100dd
3.576e+10 ~ 5% +40.4% 5.022e+10 ~ 6% fat/micro/dd-write/1HDD-cfq-ext4-100dd
7.988e+10 +20.9% 9.658e+10 TOTAL perf-stat.bus-cycles
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
2.041e+09 ~ 0% +22.2% 2.495e+09 ~ 1% fat/micro/dd-write/1HDD-cfq-ext4-100dd
2.041e+09 +22.2% 2.495e+09 TOTAL perf-stat.branch-misses
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
113694 ~ 1% -15.1% 96524 ~ 2% fat/micro/dd-write/1HDD-cfq-btrfs-100dd
115451 ~ 2% -19.4% 93077 ~ 2% fat/micro/dd-write/1HDD-cfq-ext4-100dd
229145 -17.3% 189601 TOTAL perf-stat.cpu-migrations
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
1.782e+11 ~ 4% +17.9% 2.1e+11 ~ 3% fat/micro/dd-write/1HDD-cfq-ext4-100dd
1.782e+11 +17.9% 2.1e+11 TOTAL perf-stat.dTLB-loads
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
2.345e+08 ~ 3% -9.5% 2.123e+08 ~ 3% fat/micro/dd-write/1HDD-cfq-btrfs-100dd
2.357e+08 ~ 4% -20.1% 1.883e+08 ~ 1% fat/micro/dd-write/1HDD-cfq-ext4-100dd
4.702e+08 -14.8% 4.006e+08 TOTAL perf-stat.LLC-load-misses
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
1.418e+11 ~ 2% +17.4% 1.666e+11 ~ 3% fat/micro/dd-write/1HDD-cfq-ext4-100dd
1.418e+11 +17.4% 1.666e+11 TOTAL perf-stat.L1-dcache-loads
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
1.838e+09 ~ 3% +16.1% 2.133e+09 ~ 1% fat/micro/dd-write/1HDD-cfq-ext4-100dd
1.838e+09 +16.1% 2.133e+09 TOTAL perf-stat.LLC-stores
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
1.275e+09 ~ 2% +8.3% 1.38e+09 ~ 1% fat/micro/dd-write/1HDD-cfq-btrfs-100dd
1.153e+09 ~ 2% +14.6% 1.322e+09 ~ 2% fat/micro/dd-write/1HDD-cfq-ext4-100dd
2.428e+09 +11.3% 2.702e+09 TOTAL perf-stat.LLC-loads
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
1.698e+11 ~ 1% +4.4% 1.773e+11 ~ 0% fat/micro/dd-write/1HDD-cfq-btrfs-100dd
1.09e+11 ~ 2% +15.2% 1.256e+11 ~ 2% fat/micro/dd-write/1HDD-cfq-ext4-100dd
2.788e+11 +8.6% 3.029e+11 TOTAL perf-stat.dTLB-stores
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
8.66e+08 ~ 3% +10.8% 9.594e+08 ~ 0% fat/micro/dd-write/1HDD-cfq-ext4-100dd
8.66e+08 +10.8% 9.594e+08 TOTAL perf-stat.LLC-prefetches
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
1.117e+09 ~ 2% +10.5% 1.234e+09 ~ 4% fat/micro/dd-write/1HDD-cfq-ext4-100dd
1.117e+09 +10.5% 1.234e+09 TOTAL perf-stat.dTLB-store-misses
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
1.705e+09 ~ 1% +5.3% 1.796e+09 ~ 2% fat/micro/dd-write/1HDD-cfq-ext4-100dd
1.705e+09 +5.3% 1.796e+09 TOTAL perf-stat.L1-dcache-store-misses
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
5.671e+09 ~ 2% +6.1% 6.015e+09 ~ 2% fat/micro/dd-write/1HDD-cfq-ext4-100dd
5.671e+09 +6.1% 6.015e+09 TOTAL perf-stat.L1-dcache-load-misses
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
8.794e+10 ~ 0% +3.6% 9.109e+10 ~ 2% fat/micro/dd-write/1HDD-cfq-ext4-100dd
8.794e+10 +3.6% 9.109e+10 TOTAL perf-stat.L1-dcache-stores
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
3.46e+09 ~ 1% +4.6% 3.618e+09 ~ 1% fat/micro/dd-write/1HDD-cfq-ext4-100dd
3.46e+09 +4.6% 3.618e+09 TOTAL perf-stat.cache-references
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
8.696e+08 ~ 0% +1.8% 8.849e+08 ~ 0% fat/micro/dd-write/1HDD-cfq-btrfs-100dd
8.696e+08 +1.8% 8.849e+08 TOTAL perf-stat.cache-misses
ab573844e3058ee 9e6302056f8029f438e853432
--------------- -------------------------
1613129 ~ 2% +2.6% 1655724 ~ 1% fat/micro/dd-write/1HDD-cfq-btrfs-100dd
1613129 +2.6% 1655724 TOTAL perf-stat.context-switches
perf-stat.stalled-cycles-frontend
1.2e+13 ++---------------------------------------------------------------+
| O O |
1.1e+13 ++ O O O O O O O |
1e+13 O+ O O O O O O O O O O
| |
9e+12 ++ |
8e+12 ++ |
| |
7e+12 ++ |
6e+12 ++ |
| |
5e+12 ++ |
4e+12 *+. ..*..*...*..*...*..*..*...*..*...*..*..*...*..*...*..*...*..*
| *. |
3e+12 ++---------------------------------------------------------------+
perf-stat.stalled-cycles-backend
7.5e+12 ++---------------O------O--------------------------O------O------+
7e+12 ++ O O O O O O O O |
O O O O O O O O
6.5e+12 ++ |
6e+12 ++ |
| |
5.5e+12 ++ |
5e+12 ++ |
4.5e+12 ++ |
| |
4e+12 ++ |
3.5e+12 ++ |
| .*... .*.. ..*.. |
3e+12 *+. ..*..*...*. *. *...*..*. *..*...*..*...*..*...*..*
2.5e+12 ++-*-------------------------------------------------------------+
vmstat.system.in
5500 ++------------------------------------------------------------------+
| O O |
5000 O+ O O O O O O O O O |
4500 ++ O O O O O O O O
| |
4000 ++ |
3500 ++ |
| |
3000 ++ |
2500 ++ |
| |
2000 ++ |
1500 *+..*..*... .*...*..*... ..*..*...*..*...*...*..*...*..*... .*...*
| *. *. *. |
1000 ++------------------------------------------------------------------+
perf-stat.node-prefetch-misses
9e+07 ++-----------------------------------------------------------------+
| * |
8e+07 ++ : + *.. |
7e+07 *+ *. : + : . |
| .. + .. : *.. ..*.. .*.. : * *...*
6e+07 ++ + : . .*. .. : + .. |
5e+07 ++ * *..* *. * *...* + ..* |
| *. |
4e+07 ++ |
3e+07 ++ |
| |
2e+07 ++ |
1e+07 ++ |
| |
0 O+--O--O---O--O---O--O---O--O---O--O---O--O---O--O---O--O---O------+
View attachment "dmesg" of type "text/plain" (68959 bytes)
View attachment "x86_64-lkp" of type "text/plain" (80581 bytes)
Powered by blists - more mailing lists