Message-ID: <20260108191105.695131-1-irogers@google.com>
Date: Thu,  8 Jan 2026 11:10:30 -0800
From: Ian Rogers <irogers@...gle.com>
To: Adrian Hunter <adrian.hunter@...el.com>, 
	Alexander Shishkin <alexander.shishkin@...ux.intel.com>, 
	Arnaldo Carvalho de Melo <acme@...nel.org>, Benjamin Gray <bgray@...ux.ibm.com>, 
	Caleb Biggers <caleb.biggers@...el.com>, Edward Baker <edward.baker@...el.com>, 
	Ian Rogers <irogers@...gle.com>, Ingo Molnar <mingo@...hat.com>, 
	James Clark <james.clark@...aro.org>, Jing Zhang <renyu.zj@...ux.alibaba.com>, 
	Jiri Olsa <jolsa@...nel.org>, John Garry <john.g.garry@...cle.com>, Leo Yan <leo.yan@....com>, 
	Namhyung Kim <namhyung@...nel.org>, Perry Taylor <perry.taylor@...el.com>, 
	Peter Zijlstra <peterz@...radead.org>, Samantha Alt <samantha.alt@...el.com>, 
	Sandipan Das <sandipan.das@....com>, Thomas Falcon <thomas.falcon@...el.com>, 
	Weilin Wang <weilin.wang@...el.com>, Xu Yang <xu.yang_2@....com>, linux-kernel@...r.kernel.org, 
	linux-perf-users@...r.kernel.org
Subject: [PATCH v10 00/35] AMD and Intel metric generation with Python

Metrics in the perf tool come in via json. Json doesn't allow
comments, line breaks, etc. making it an inconvenient way to write
metrics. Further, it is useful to detect when writing a metric that
the event specified is supported within the event json for a
model. From the metric python code Event(s) are used, with fallback
events provided, if no event is found then an exception is thrown and
that can either indicate a failure or an unsupported model. To avoid
confusion all the metrics and their metricgroups are prefixed with
'lpm_', where LPM is an abbreviation of Linux Perf Metric. While extra
characters aren't ideal, this separates the metrics from other vendor
provided metrics.

* The first 2 patches introduce infrastructure for the addition of
  metrics written in python for Arm64, AMD Zen and Intel CPUs.

* The next 9 patches generate additional metrics for AMD zen.  Rapl
  and Idle metrics aren't specific to AMD but are placed here for ease
  and convenience. Uncore L3 metrics are added along with the majority
  of core metrics.

* The next 22 patches add additional metrics for Intel. Rapl and Idle
  metrics aren't specific to Intel but are placed here for ease and
  convenience. Smi and tsx metrics are added so they can be dropped
  from the per model json files. There are four uncore sets of metrics
  and eleven core metrics. Add a CheckPmu function to metric to
  simplify detecting the presence of hybrid PMUs in events. Metrics
  with experimental events are flagged as experimental in their
  description.

* The next patch adds a cycles metrics based on perf event modifiers
  for AMD, Intel and Arm64.

* The final patch validates that all events provided to an Event
  object exist in a json file somewhere. This is to avoid mistakes
  like unfortunate typos.

This series has benefitted from the input of Leo Yan
 <leo.yan@....com>, Sandipan Das <sandidas@....com>, Thomas Falcon
 <thomas.falcon@...el.com> and Perry Taylor <perry.taylor@...el.com>.

v10. Drop already merged non-vendor patches (Namhyung). Drop "Add
     collection of topdown like metrics for arm64" as requested by
     James Clark. Update AMD metrics for changes to AMD Zen6 event
     names from the series:
     https://lore.kernel.org/lkml/cover.1767858676.git.sandipan.das@amd.com/

v9. Drop (for now) 4 AMD sets of metrics for additional follow up. Add
    reviewed-by tags from Sandipan Das (AMD) and tested-by tags from
    Thomas Falcon (Intel).
    https://lore.kernel.org/lkml/20251202175043.623597-1-irogers@google.com/

v8. Combine the previous 4 series for clarity. Rebase on top of the
    more recent legacy metric and event changes. Make the python more
    pep8 and pylint compliant.
    https://lore.kernel.org/lkml/20251113032040.1994090-1-irogers@google.com/

Foundations:
v6. Fix issue with '\-' escape not being '\\-' (reported-by Sandipan
    Das <sandidas@....com>) which didn't alter the generated json.
    https://lore.kernel.org/lkml/20250904043208.995243-1-irogers@google.com/

v5. Rebase on top of legacy hardware/cache changes that now generate
    events using python:
    https://lore.kernel.org/lkml/20250828205930.4007284-1-irogers@google.com/
    the v5 series is:
    https://lore.kernel.org/lkml/20250829030727.4159703-1-irogers@google.com/

v4. Rebase and small Build/Makefile tweak
    https://lore.kernel.org/lkml/20240926173554.404411-1-irogers@google.com/

v3. Some code tidying, make the input directory a command line
    argument, but no other functional or output changes.
    https://lore.kernel.org/lkml/20240314055051.1960527-1-irogers@google.com/

v2. Fixes two type issues in the python code but no functional or
    output changes.
    https://lore.kernel.org/lkml/20240302005950.2847058-1-irogers@google.com/

v1. https://lore.kernel.org/lkml/20240302005950.2847058-1-irogers@google.com/

AMD:
v6. Fix issue with '\-' escape not being '\\-' (reported-by Sandipan
    Das <sandidas@....com>) which didn't alter the generated json.
    https://lore.kernel.org/lkml/20250904044047.999031-1-irogers@google.com/

v5. Rebase. Add uop cache hit/miss rates patch. Prefix all metric
    names with lpm_ (short for Linux Perf Metric) so that python
    generated metrics are clearly namespaced.
    https://lore.kernel.org/lkml/20250829033138.4166591-1-irogers@google.com/

v4. Rebase.
    https://lore.kernel.org/lkml/20240926174101.406874-1-irogers@google.com/

v3. Some minor code cleanup changes.
    https://lore.kernel.org/lkml/20240314055839.1975063-1-irogers@google.com/

v2. Drop the cycles breakdown in favor of having it as a common
    metric, suggested by Kan Liang <kan.liang@...ux.intel.com>.
    https://lore.kernel.org/lkml/20240301184737.2660108-1-irogers@google.com/

v1. https://lore.kernel.org/lkml/20240229001537.4158049-1-irogers@google.com/

Intel:
v6. Fix issue with '\-' escape not being '\\-' (reported-by Sandipan
    Das <sandidas@....com>) which didn't alter the generated json.
    https://lore.kernel.org/lkml/20250904044653.1002362-1-irogers@google.com/

v5. Rebase. Fix description for smi metric (Kan). Prefix all metric
    names with lpm_ (short for Linux Perf Metric) so that python
    generated metrics are clearly namespaced. Kan requested a
    namespace in his review:
    https://lore.kernel.org/lkml/43548903-b7c8-47c4-b1da-0258293ecbd4@linux.intel.com/
    The v5 series is:
    https://lore.kernel.org/lkml/20250829041104.4186320-1-irogers@google.com/

v4. Experimental metric descriptions. Add mesh bandwidth metric. Rebase.
    https://lore.kernel.org/lkml/20240926175035.408668-1-irogers@google.com/

v3. Swap tsx and CheckPMU patches that were in the wrong order. Some
    minor code cleanup changes. Drop reference to merged fix for
    umasks/occ_sel in PCU events and for cstate metrics.
    https://lore.kernel.org/lkml/20240314055919.1979781-1-irogers@google.com/

v2. Drop the cycles breakdown in favor of having it as a common
    metric, spelling and other improvements suggested by Kan Liang
    <kan.liang@...ux.intel.com>.
    https://lore.kernel.org/lkml/20240301185559.2661241-1-irogers@google.com/

v1. https://lore.kernel.org/lkml/20240229001806.4158429-1-irogers@google.com/

ARM:

v7. Switch a use of cycles to cpu-cycles due to ARM having too many
    cycles events.
    https://lore.kernel.org/lkml/20250904194139.1540230-1-irogers@google.com/

v6. Fix issue with '\-' escape not being '\\-' (reported-by Sandipan
    Das <sandidas@....com>) which didn't alter the generated json.
    https://lore.kernel.org/lkml/20250904045253.1007052-1-irogers@google.com/

v5. Rebase. Address review comments from Leo Yan
    <leo.yan@....com>. Prefix all metric names with lpm_ (short for
    Linux Perf Metric) so that python generated metrics are clearly
    namespaced. Use cpu-cycles rather than cycles legacy event for
    cycles metrics to avoid confusion with ARM PMUs. Add patch that
    checks events to ensure all possible event names are present in at
    least one json file.
    https://lore.kernel.org/lkml/20250829053235.21994-1-irogers@google.com/

v4. Tweak to build dependencies and rebase.
    https://lore.kernel.org/lkml/20240926175709.410022-1-irogers@google.com/

v3. Some minor code cleanup changes.
    https://lore.kernel.org/lkml/20240314055801.1973422-1-irogers@google.com/

v2. The cycles metrics are now made common and shared with AMD and
    Intel, suggested by Kan Liang <kan.liang@...ux.intel.com>. This
    assumes these patches come after the AMD and Intel sets.
    https://lore.kernel.org/lkml/20240301184942.2660478-1-irogers@google.com/

v1. https://lore.kernel.org/lkml/20240229001325.4157655-1-irogers@google.com/

Ian Rogers (35):
  perf jevents: Build support for generating metrics from python
  perf jevents: Add load event json to verify and allow fallbacks
  perf jevents: Add RAPL event metric for AMD zen models
  perf jevents: Add idle metric for AMD zen models
  perf jevents: Add upc metric for uops per cycle for AMD
  perf jevents: Add br metric group for branch statistics on AMD
  perf jevents: Add itlb metric group for AMD
  perf jevents: Add dtlb metric group for AMD
  perf jevents: Add uncore l3 metric group for AMD
  perf jevents: Add load store breakdown metrics ldst for AMD
  perf jevents: Add context switch metrics for AMD
  perf jevents: Add RAPL metrics for all Intel models
  perf jevents: Add idle metric for Intel models
  perf jevents: Add CheckPmu to see if a PMU is in loaded json events
  perf jevents: Add smi metric group for Intel models
  perf jevents: Mark metrics with experimental events as experimental
  perf jevents: Add tsx metric group for Intel models
  perf jevents: Add br metric group for branch statistics on Intel
  perf jevents: Add software prefetch (swpf) metric group for Intel
  perf jevents: Add ports metric group giving utilization on Intel
  perf jevents: Add L2 metrics for Intel
  perf jevents: Add load store breakdown metrics ldst for Intel
  perf jevents: Add ILP metrics for Intel
  perf jevents: Add context switch metrics for Intel
  perf jevents: Add FPU metrics for Intel
  perf jevents: Add Miss Level Parallelism (MLP) metric for Intel
  perf jevents: Add mem_bw metric for Intel
  perf jevents: Add local/remote "mem" breakdown metrics for Intel
  perf jevents: Add dir breakdown metrics for Intel
  perf jevents: Add C-State metrics from the PCU PMU for Intel
  perf jevents: Add local/remote miss latency metrics for Intel
  perf jevents: Add upi_bw metric for Intel
  perf jevents: Add mesh bandwidth saturation metric for Intel
  perf jevents: Add cycles breakdown metric for arm64/AMD/Intel
  perf jevents: Validate that all names given an Event

 tools/perf/.gitignore                   |    5 +
 tools/perf/Makefile.perf                |    2 +
 tools/perf/pmu-events/Build             |   51 +-
 tools/perf/pmu-events/amd_metrics.py    |  492 ++++++++++
 tools/perf/pmu-events/arm64_metrics.py  |   49 +
 tools/perf/pmu-events/common_metrics.py |   19 +
 tools/perf/pmu-events/intel_metrics.py  | 1129 +++++++++++++++++++++++
 tools/perf/pmu-events/metric.py         |  171 +++-
 8 files changed, 1914 insertions(+), 4 deletions(-)
 create mode 100755 tools/perf/pmu-events/amd_metrics.py
 create mode 100755 tools/perf/pmu-events/arm64_metrics.py
 create mode 100644 tools/perf/pmu-events/common_metrics.py
 create mode 100755 tools/perf/pmu-events/intel_metrics.py

-- 
2.52.0.457.g6b5491de43-goog

Powered by blists - more mailing lists