Message-ID: <CAP-5=fUY2DZJ=6DDig-zHQ+G6CU0K4EGVSAsx6E5XLbdc40tLw@mail.gmail.com>
Date: Fri, 23 Jan 2026 09:12:20 -0800
From: Ian Rogers <irogers@...gle.com>
To: Adrian Hunter <adrian.hunter@...el.com>, 
	Alexander Shishkin <alexander.shishkin@...ux.intel.com>, 
	Arnaldo Carvalho de Melo <acme@...nel.org>, Benjamin Gray <bgray@...ux.ibm.com>, 
	Caleb Biggers <caleb.biggers@...el.com>, Edward Baker <edward.baker@...el.com>, 
	Ian Rogers <irogers@...gle.com>, Ingo Molnar <mingo@...hat.com>, 
	James Clark <james.clark@...aro.org>, Jing Zhang <renyu.zj@...ux.alibaba.com>, 
	Jiri Olsa <jolsa@...nel.org>, John Garry <john.g.garry@...cle.com>, Leo Yan <leo.yan@....com>, 
	Namhyung Kim <namhyung@...nel.org>, Perry Taylor <perry.taylor@...el.com>, 
	Peter Zijlstra <peterz@...radead.org>, Samantha Alt <samantha.alt@...el.com>, 
	Sandipan Das <sandipan.das@....com>, Thomas Falcon <thomas.falcon@...el.com>, 
	Weilin Wang <weilin.wang@...el.com>, Xu Yang <xu.yang_2@....com>, linux-kernel@...r.kernel.org, 
	linux-perf-users@...r.kernel.org
Subject: Re: [PATCH v10 00/35] AMD and Intel metric generation with Python

On Mon, Jan 19, 2026 at 9:23 PM Ian Rogers <irogers@...gle.com> wrote:
>
> On Thu, Jan 8, 2026 at 11:11 AM Ian Rogers <irogers@...gle.com> wrote:
> >
> > Metrics in the perf tool come in via json. Json doesn't allow
> > comments, line breaks, etc. making it an inconvenient way to write
> > metrics. Further, it is useful to detect when writing a metric that
> > the event specified is supported within the event json for a
> > model. From the metric python code Event(s) are used, with fallback
> > events provided, if no event is found then an exception is thrown and
> > that can either indicate a failure or an unsupported model. To avoid
> > confusion all the metrics and their metricgroups are prefixed with
> > 'lpm_', where LPM is an abbreviation of Linux Perf Metric. While extra
> > characters aren't ideal, this separates the metrics from other vendor
> > provided metrics.
> >
> > * The first 2 patches introduce infrastructure for the addition of
> >   metrics written in python for Arm64, AMD Zen and Intel CPUs.
> >
> > * The next 9 patches generate additional metrics for AMD zen.  Rapl
> >   and Idle metrics aren't specific to AMD but are placed here for ease
> >   and convenience. Uncore L3 metrics are added along with the majority
> >   of core metrics.
> >
> > * The next 22 patches add additional metrics for Intel. Rapl and Idle
> >   metrics aren't specific to Intel but are placed here for ease and
> >   convenience. Smi and tsx metrics are added so they can be dropped
> >   from the per model json files. There are four uncore sets of metrics
> >   and eleven core metrics. Add a CheckPmu function to metric to
> >   simplify detecting the presence of hybrid PMUs in events. Metrics
> >   with experimental events are flagged as experimental in their
> >   description.
> >
> > * The next patch adds a cycles metrics based on perf event modifiers
> >   for AMD, Intel and Arm64.
> >
> > * The final patch validates that all events provided to an Event
> >   object exist in a json file somewhere. This is to avoid mistakes
> >   like unfortunate typos.
> >
> > This series has benefitted from the input of Leo Yan
> >  <leo.yan@....com>, Sandipan Das <sandidas@....com>, Thomas Falcon
> >  <thomas.falcon@...el.com> and Perry Taylor <perry.taylor@...el.com>.
> >
> > v10. Drop already merged non-vendor patches (Namhyung). Drop "Add
> >      collection of topdown like metrics for arm64" as requested by
> >      James Clark. Update AMD metrics for changes to AMD Zen6 event
> >      names from the series:
> >      https://lore.kernel.org/lkml/cover.1767858676.git.sandipan.das@amd.com/
> >
> > v9. Drop (for now) 4 AMD sets of metrics for additional follow up. Add
> >     reviewed-by tags from Sandipan Das (AMD) and tested-by tags from
> >     Thomas Falcon (Intel).
> >     https://lore.kernel.org/lkml/20251202175043.623597-1-irogers@google.com/
> >
> > v8. Combine the previous 4 series for clarity. Rebase on top of the
> >     more recent legacy metric and event changes. Make the python more
> >     pep8 and pylint compliant.
> >     https://lore.kernel.org/lkml/20251113032040.1994090-1-irogers@google.com/
> >
> > Foundations:
> > v6. Fix issue with '\-' escape not being '\\-' (reported-by Sandipan
> >     Das <sandidas@....com>) which didn't alter the generated json.
> >     https://lore.kernel.org/lkml/20250904043208.995243-1-irogers@google.com/
> >
> > v5. Rebase on top of legacy hardware/cache changes that now generate
> >     events using python:
> >     https://lore.kernel.org/lkml/20250828205930.4007284-1-irogers@google.com/
> >     the v5 series is:
> >     https://lore.kernel.org/lkml/20250829030727.4159703-1-irogers@google.com/
> >
> > v4. Rebase and small Build/Makefile tweak
> >     https://lore.kernel.org/lkml/20240926173554.404411-1-irogers@google.com/
> >
> > v3. Some code tidying, make the input directory a command line
> >     argument, but no other functional or output changes.
> >     https://lore.kernel.org/lkml/20240314055051.1960527-1-irogers@google.com/
> >
> > v2. Fixes two type issues in the python code but no functional or
> >     output changes.
> >     https://lore.kernel.org/lkml/20240302005950.2847058-1-irogers@google.com/
> >
> > v1. https://lore.kernel.org/lkml/20240302005950.2847058-1-irogers@google.com/
> >
> > AMD:
> > v6. Fix issue with '\-' escape not being '\\-' (reported-by Sandipan
> >     Das <sandidas@....com>) which didn't alter the generated json.
> >     https://lore.kernel.org/lkml/20250904044047.999031-1-irogers@google.com/
> >
> > v5. Rebase. Add uop cache hit/miss rates patch. Prefix all metric
> >     names with lpm_ (short for Linux Perf Metric) so that python
> >     generated metrics are clearly namespaced.
> >     https://lore.kernel.org/lkml/20250829033138.4166591-1-irogers@google.com/
> >
> > v4. Rebase.
> >     https://lore.kernel.org/lkml/20240926174101.406874-1-irogers@google.com/
> >
> > v3. Some minor code cleanup changes.
> >     https://lore.kernel.org/lkml/20240314055839.1975063-1-irogers@google.com/
> >
> > v2. Drop the cycles breakdown in favor of having it as a common
> >     metric, suggested by Kan Liang <kan.liang@...ux.intel.com>.
> >     https://lore.kernel.org/lkml/20240301184737.2660108-1-irogers@google.com/
> >
> > v1. https://lore.kernel.org/lkml/20240229001537.4158049-1-irogers@google.com/
> >
> > Intel:
> > v6. Fix issue with '\-' escape not being '\\-' (reported-by Sandipan
> >     Das <sandidas@....com>) which didn't alter the generated json.
> >     https://lore.kernel.org/lkml/20250904044653.1002362-1-irogers@google.com/
> >
> > v5. Rebase. Fix description for smi metric (Kan). Prefix all metric
> >     names with lpm_ (short for Linux Perf Metric) so that python
> >     generated metrics are clearly namespaced. Kan requested a
> >     namespace in his review:
> >     https://lore.kernel.org/lkml/43548903-b7c8-47c4-b1da-0258293ecbd4@linux.intel.com/
> >     The v5 series is:
> >     https://lore.kernel.org/lkml/20250829041104.4186320-1-irogers@google.com/
> >
> > v4. Experimental metric descriptions. Add mesh bandwidth metric. Rebase.
> >     https://lore.kernel.org/lkml/20240926175035.408668-1-irogers@google.com/
> >
> > v3. Swap tsx and CheckPMU patches that were in the wrong order. Some
> >     minor code cleanup changes. Drop reference to merged fix for
> >     umasks/occ_sel in PCU events and for cstate metrics.
> >     https://lore.kernel.org/lkml/20240314055919.1979781-1-irogers@google.com/
> >
> > v2. Drop the cycles breakdown in favor of having it as a common
> >     metric, spelling and other improvements suggested by Kan Liang
> >     <kan.liang@...ux.intel.com>.
> >     https://lore.kernel.org/lkml/20240301185559.2661241-1-irogers@google.com/
> >
> > v1. https://lore.kernel.org/lkml/20240229001806.4158429-1-irogers@google.com/
> >
> > ARM:
> >
> > v7. Switch a use of cycles to cpu-cycles due to ARM having too many
> >     cycles events.
> >     https://lore.kernel.org/lkml/20250904194139.1540230-1-irogers@google.com/
> >
> > v6. Fix issue with '\-' escape not being '\\-' (reported-by Sandipan
> >     Das <sandidas@....com>) which didn't alter the generated json.
> >     https://lore.kernel.org/lkml/20250904045253.1007052-1-irogers@google.com/
> >
> > v5. Rebase. Address review comments from Leo Yan
> >     <leo.yan@....com>. Prefix all metric names with lpm_ (short for
> >     Linux Perf Metric) so that python generated metrics are clearly
> >     namespaced. Use cpu-cycles rather than cycles legacy event for
> >     cycles metrics to avoid confusion with ARM PMUs. Add patch that
> >     checks events to ensure all possible event names are present in at
> >     least one json file.
> >     https://lore.kernel.org/lkml/20250829053235.21994-1-irogers@google.com/
> >
> > v4. Tweak to build dependencies and rebase.
> >     https://lore.kernel.org/lkml/20240926175709.410022-1-irogers@google.com/
> >
> > v3. Some minor code cleanup changes.
> >     https://lore.kernel.org/lkml/20240314055801.1973422-1-irogers@google.com/
> >
> > v2. The cycles metrics are now made common and shared with AMD and
> >     Intel, suggested by Kan Liang <kan.liang@...ux.intel.com>. This
> >     assumes these patches come after the AMD and Intel sets.
> >     https://lore.kernel.org/lkml/20240301184942.2660478-1-irogers@google.com/
> >
> > v1. https://lore.kernel.org/lkml/20240229001325.4157655-1-irogers@google.com/
> >
> > Ian Rogers (35):
> >   perf jevents: Build support for generating metrics from python
> >   perf jevents: Add load event json to verify and allow fallbacks
> >   perf jevents: Add RAPL event metric for AMD zen models
> >   perf jevents: Add idle metric for AMD zen models
> >   perf jevents: Add upc metric for uops per cycle for AMD
> >   perf jevents: Add br metric group for branch statistics on AMD
> >   perf jevents: Add itlb metric group for AMD
> >   perf jevents: Add dtlb metric group for AMD
> >   perf jevents: Add uncore l3 metric group for AMD
> >   perf jevents: Add load store breakdown metrics ldst for AMD
> >   perf jevents: Add context switch metrics for AMD
> >   perf jevents: Add RAPL metrics for all Intel models
> >   perf jevents: Add idle metric for Intel models
> >   perf jevents: Add CheckPmu to see if a PMU is in loaded json events
> >   perf jevents: Add smi metric group for Intel models
> >   perf jevents: Mark metrics with experimental events as experimental
> >   perf jevents: Add tsx metric group for Intel models
> >   perf jevents: Add br metric group for branch statistics on Intel
> >   perf jevents: Add software prefetch (swpf) metric group for Intel
> >   perf jevents: Add ports metric group giving utilization on Intel
> >   perf jevents: Add L2 metrics for Intel
> >   perf jevents: Add load store breakdown metrics ldst for Intel
> >   perf jevents: Add ILP metrics for Intel
> >   perf jevents: Add context switch metrics for Intel
> >   perf jevents: Add FPU metrics for Intel
> >   perf jevents: Add Miss Level Parallelism (MLP) metric for Intel
> >   perf jevents: Add mem_bw metric for Intel
> >   perf jevents: Add local/remote "mem" breakdown metrics for Intel
> >   perf jevents: Add dir breakdown metrics for Intel
> >   perf jevents: Add C-State metrics from the PCU PMU for Intel
> >   perf jevents: Add local/remote miss latency metrics for Intel
> >   perf jevents: Add upi_bw metric for Intel
> >   perf jevents: Add mesh bandwidth saturation metric for Intel
> >   perf jevents: Add cycles breakdown metric for arm64/AMD/Intel
> >   perf jevents: Validate that all names given an Event
>
> Hi Arnaldo and Namhyung,
>
> this patch series has carved out everything anybody has objected to
> and every patch has tags. I'd appreciate it landing. The v1 series was
> originally mailed in February 2024 :-)

Ping.

Thanks,
Ian

> Thanks,
> Ian
>
> >  tools/perf/.gitignore                   |    5 +
> >  tools/perf/Makefile.perf                |    2 +
> >  tools/perf/pmu-events/Build             |   51 +-
> >  tools/perf/pmu-events/amd_metrics.py    |  492 ++++++++++
> >  tools/perf/pmu-events/arm64_metrics.py  |   49 +
> >  tools/perf/pmu-events/common_metrics.py |   19 +
> >  tools/perf/pmu-events/intel_metrics.py  | 1129 +++++++++++++++++++++++
> >  tools/perf/pmu-events/metric.py         |  171 +++-
> >  8 files changed, 1914 insertions(+), 4 deletions(-)
> >  create mode 100755 tools/perf/pmu-events/amd_metrics.py
> >  create mode 100755 tools/perf/pmu-events/arm64_metrics.py
> >  create mode 100644 tools/perf/pmu-events/common_metrics.py
> >  create mode 100755 tools/perf/pmu-events/intel_metrics.py
> >
> > --
> > 2.52.0.457.g6b5491de43-goog
> >

Powered by blists - more mailing lists