| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <20251024175857.808401-1-irogers@google.com>
Date: Fri, 24 Oct 2025 10:58:35 -0700
From: Ian Rogers <irogers@...gle.com>
To: Peter Zijlstra <peterz@...radead.org>, Ingo Molnar <mingo@...hat.com>,
Arnaldo Carvalho de Melo <acme@...nel.org>, Namhyung Kim <namhyung@...nel.org>,
Alexander Shishkin <alexander.shishkin@...ux.intel.com>, Jiri Olsa <jolsa@...nel.org>,
Ian Rogers <irogers@...gle.com>, Adrian Hunter <adrian.hunter@...el.com>,
James Clark <james.clark@...aro.org>, Xu Yang <xu.yang_2@....com>,
Chun-Tse Shao <ctshao@...gle.com>, Thomas Richter <tmricht@...ux.ibm.com>,
Sumanth Korikkar <sumanthk@...ux.ibm.com>, Collin Funk <collin.funk1@...il.com>,
Thomas Falcon <thomas.falcon@...el.com>, Howard Chu <howardchu95@...il.com>,
Dapeng Mi <dapeng1.mi@...ux.intel.com>, Levi Yun <yeoreum.yun@....com>,
Yang Li <yang.lee@...ux.alibaba.com>, linux-kernel@...r.kernel.org,
linux-perf-users@...r.kernel.org
Subject: [PATCH v1 00/22] Switch the default perf stat metrics to json
Prior to this series stat-shadow would produce hard coded metrics if
certain events appeared in the evlist. This series produces equivalent
json metrics and cleans up the consequences in tests and display
output. A before and after of the default display output on a
tigerlake is:
Before:
```
$ perf stat -a sleep 1
Performance counter stats for 'system wide':
16,041,816,418 cpu-clock # 15.995 CPUs utilized
5,749 context-switches # 358.376 /sec
121 cpu-migrations # 7.543 /sec
1,806 page-faults # 112.581 /sec
825,965,204 instructions # 0.70 insn per cycle
1,180,799,101 cycles # 0.074 GHz
168,945,109 branches # 10.532 M/sec
4,629,567 branch-misses # 2.74% of all branches
# 30.2 % tma_backend_bound
# 7.8 % tma_bad_speculation
# 47.1 % tma_frontend_bound
# 14.9 % tma_retiring
```
After:
```
$ perf stat -a sleep 1
Performance counter stats for 'system wide':
2,890 context-switches # 179.9 cs/sec cs_per_second
16,061,923,339 cpu-clock # 16.0 CPUs CPUs_utilized
43 cpu-migrations # 2.7 migrations/sec migrations_per_second
5,645 page-faults # 351.5 faults/sec page_faults_per_second
5,708,413 branch-misses # 1.4 % branch_miss_rate (88.83%)
429,978,120 branches # 26.8 K/sec branch_frequency (88.85%)
1,626,915,897 cpu-cycles # 0.1 GHz cycles_frequency (88.84%)
2,556,805,534 instructions # 1.5 instructions insn_per_cycle (88.86%)
TopdownL1 # 20.1 % tma_backend_bound
# 40.5 % tma_bad_speculation (88.90%)
# 17.2 % tma_frontend_bound (78.05%)
# 22.2 % tma_retiring (88.89%)
1.002994394 seconds time elapsed
```
Having the metrics in json brings greater uniformity, allows events to
be shared by metrics, and it also allows descriptions like:
```
$ perf list cs_per_second
...
cs_per_second
[Context switches per CPU second]
```
A thorn in the side of doing this work was that the hard coded metrics
were used by perf script with '-F metric'. This functionality didn't
work for me (I was testing `perf record -e instructions,cycles` and
then `perf script -F metric` but saw nothing but empty lines) but
anyway I decided to fix it to the best of my ability in this
series. So the script side counters were removed and the regular ones
associated with the evsel used. The json metrics were all searched
looking for ones that have a subset of events matching those in the
perf script session, and all metrics are printed. This is kind of
weird as the counters are being set by the period of samples, but I
carried the behavior forward. I suspect there needs to be follow up
work to make this better, but what is in the series is superior to
what is currently in the tree. Follow up work could include finding
metrics for the machine in the perf.data rather than using the host,
allowing multiple metrics even if the metric ids of the events differ,
fixing pre-existing `perf stat record/report` issues, etc.
There is a lot of stat tests that, for example, assume '-e
instructions,cycles' will produce an IPC metric. These things needed
tidying as now the metric must be explicitly asked for and when doing
this ones using software events were preferred to increase
compatibility. As the test updates were numerous they are distinct to
the patches updating the functionality causing periods in the series
where not all tests are passing. If this is undesirable the test fixes
can be squashed into the functionality updates.
Ian Rogers (22):
perf evsel: Remove unused metric_events variable
perf metricgroup: Update comment on location of metric_event list
perf metricgroup: Missed free on error path
perf metricgroup: When copy metrics copy default information
perf metricgroup: Add care to picking the evsel for displaying a
metric
perf jevents: Make all tables static
perf expr: Add #target_cpu literal
perf jevents: Add set of common metrics based on default ones
perf jevents: Add metric DefaultShowEvents
perf stat: Add detail -d,-dd,-ddd metrics
perf script: Change metric format to use json metrics
perf stat: Remove hard coded shadow metrics
perf stat: Fix default metricgroup display on hybrid
perf stat: Sort default events/metrics
perf stat: Remove "unit" workarounds for metric-only
perf test stat+json: Improve metric-only testing
perf test stat: Ignore failures in Default[234] metricgroups
perf test stat: Update std_output testing metric expectations
perf test metrics: Update all metrics for possibly failing default
metrics
perf test stat: Update shadow test to use metrics
perf test stat: Update test expectations and events
perf test stat csv: Update test expectations and events
tools/perf/builtin-script.c | 238 ++++++++++-
tools/perf/builtin-stat.c | 154 ++-----
.../arch/common/common/metrics.json | 151 +++++++
tools/perf/pmu-events/empty-pmu-events.c | 139 ++++--
tools/perf/pmu-events/jevents.py | 34 +-
tools/perf/pmu-events/pmu-events.h | 2 +
.../tests/shell/lib/perf_json_output_lint.py | 4 +-
tools/perf/tests/shell/lib/stat_output.sh | 2 +-
tools/perf/tests/shell/stat+csv_output.sh | 2 +-
tools/perf/tests/shell/stat+json_output.sh | 2 +-
tools/perf/tests/shell/stat+shadow_stat.sh | 4 +-
tools/perf/tests/shell/stat+std_output.sh | 4 +-
tools/perf/tests/shell/stat.sh | 6 +-
.../perf/tests/shell/stat_all_metricgroups.sh | 3 +
tools/perf/tests/shell/stat_all_metrics.sh | 7 +-
tools/perf/util/evsel.c | 2 -
tools/perf/util/evsel.h | 2 +-
tools/perf/util/expr.c | 3 +
tools/perf/util/metricgroup.c | 95 ++++-
tools/perf/util/metricgroup.h | 2 +-
tools/perf/util/stat-display.c | 55 +--
tools/perf/util/stat-shadow.c | 402 +-----------------
tools/perf/util/stat.h | 2 +-
23 files changed, 672 insertions(+), 643 deletions(-)
create mode 100644 tools/perf/pmu-events/arch/common/common/metrics.json
--
2.51.1.821.gb6fe4d2222-goog
Powered by blists - more mailing lists