linux-kernel - Re: [PATCH v4] perf script python: Add the ins

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <ZrYbbZVE4BQ-oqba@x1>
Date: Fri, 9 Aug 2024 10:36:45 -0300
From: Arnaldo Carvalho de Melo <acme@...nel.org>
To: Zixian Cai <fzczx123@...il.com>
Cc: Adrian Hunter <adrian.hunter@...el.com>,
	Peter Zijlstra <peterz@...radead.org>,
	Ingo Molnar <mingo@...hat.com>, Namhyung Kim <namhyung@...nel.org>,
	Mark Rutland <mark.rutland@....com>,
	Alexander Shishkin <alexander.shishkin@...ux.intel.com>,
	Jiri Olsa <jolsa@...nel.org>, Ian Rogers <irogers@...gle.com>,
	"Liang, Kan" <kan.liang@...ux.intel.com>,
	Ben Gainey <ben.gainey@....com>, Paran Lee <p4ranlee@...il.com>,
	linux-perf-users@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH v4] perf script python: Add the ins_lat field to event
 handler

On Fri, Aug 09, 2024 at 08:01:36AM +0000, Zixian Cai wrote:
> For example, when using the Alder Lake PMU memory load event, the
> instruction latency is stored in ins_lat, while the cache latency
> is stored in weight.
> 
> This patch reports the ins_lat field for Python scripting.

So, how did you test this? I tried:

Committer testing:

On a Rocket Lake Refresh Intel machine (14th gen):

  root@...ber:~# grep -m1 'model name' /proc/cpuinfo
  model name    : Intel(R) Core(TM) i7-14700K
  root@...ber:~# perf mem record -a sleep 5
  Memory events are enabled on a subset of CPUs: 16-27
  [ perf record: Woken up 85 times to write data ]
  [ perf record: Captured and wrote 41.236 MB perf.data (191390 samples) ]
  root@...ber:~# perf evlist -v
  cpu_atom/mem-loads,ldlat=30/P: type: 10 (cpu_atom), size: 136, config: 0x5d0 (mem-loads), { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|ADDR|CPU|PERIOD|IDENTIFIER|DATA_SRC|WEIGHT_STRUCT, read_format: ID|LOST, disabled: 1, inherit: 1, freq: 1, precise_ip: 3, sample_id_all: 1, { bp_addr, config1 }: 0x1f
  cpu_atom/mem-stores/P: type: 10 (cpu_atom), size: 136, config: 0x6d0 (mem-stores), { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|ADDR|CPU|PERIOD|IDENTIFIER|DATA_SRC|WEIGHT_STRUCT, read_format: ID|LOST, disabled: 1, inherit: 1, freq: 1, precise_ip: 3, sample_id_all: 1
  dummy:u: type: 1 (software), size: 136, config: 0x9 (PERF_COUNT_SW_DUMMY), { sample_period, sample_freq }: 1, sample_type: IP|TID|TIME|ADDR|CPU|IDENTIFIER|DATA_SRC|WEIGHT_STRUCT, read_format: ID|LOST, inherit: 1, exclude_kernel: 1, exclude_hv: 1, mmap: 1, comm: 1, task: 1, mmap_data: 1, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1, ksymbol: 1, bpf_event: 1
  root@...ber:~#

Now generate a python script to then dump the dictionary that now needs
to have that 'ins_lat' field:

  root@...ber:~# perf script --gen python
  generated Python script: perf-script.py
  root@...ber:~# vim perf-script.py
  root@...ber:~# perf script -s perf-script.py | head -40
  in trace_begin
  in trace_end
  root@...ber:~# vim perf-script.py

But now the perf-script.py doesn't have a handler for the events and I
got just:

  root@...ber:~# perf script -s perf-script.py 
  in trace_begin
  in trace_end
  root@...ber:~# perf evlist 
  cpu_atom/mem-loads,ldlat=30/P
  cpu_atom/mem-stores/P
  dummy:u
  root@...ber:~# perf report -D | grep PERF_RECORD_SAMPLE | wc -l
  5857
  root@...ber:~#

So now I'm investigating if this is some 'perf script' script generation
oddity by trying to run this on an AMD machine, non-hybrid...

But in general try to provide the steps to show that the functionality
that you are adding is actually working, making it easy for other
people to try reproducing your results.

Thanks,

- Arnaldo
 
> Signed-off-by: Zixian Cai <fzczx123@...il.com>
> Reviewed-by: Adrian Hunter <adrian.hunter@...el.com>
> ---
> v4) reformat commit message for max line length
> v3) address review comments
> v2) rebase on top of perf-tools-next
> 
>  tools/perf/util/scripting-engines/trace-event-python.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/perf/util/scripting-engines/trace-event-python.c b/tools/perf/util/scripting-engines/trace-event-python.c
> index fb00f3ad6815..6971dd6c231f 100644
> --- a/tools/perf/util/scripting-engines/trace-event-python.c
> +++ b/tools/perf/util/scripting-engines/trace-event-python.c
> @@ -888,6 +888,8 @@ static PyObject *get_perf_sample_dict(struct perf_sample *sample,
>  	set_sample_read_in_dict(dict_sample, sample, evsel);
>  	pydict_set_item_string_decref(dict_sample, "weight",
>  			PyLong_FromUnsignedLongLong(sample->weight));
> +	pydict_set_item_string_decref(dict_sample, "ins_lat",
> +			PyLong_FromUnsignedLong(sample->ins_lat));
>  	pydict_set_item_string_decref(dict_sample, "transaction",
>  			PyLong_FromUnsignedLongLong(sample->transaction));
>  	set_sample_datasrc_in_dict(dict_sample, sample);
> @@ -1317,7 +1319,7 @@ static void python_export_sample_table(struct db_export *dbe,
>  	struct tables *tables = container_of(dbe, struct tables, dbe);
>  	PyObject *t;
> 
> -	t = tuple_new(27);
> +	t = tuple_new(28);
> 
>  	tuple_set_d64(t, 0, es->db_id);
>  	tuple_set_d64(t, 1, es->evsel->db_id);
> @@ -1346,6 +1348,7 @@ static void python_export_sample_table(struct db_export *dbe,
>  	tuple_set_s32(t, 24, es->sample->flags);
>  	tuple_set_d64(t, 25, es->sample->id);
>  	tuple_set_d64(t, 26, es->sample->stream_id);
> +	tuple_set_u32(t, 27, es->sample->ins_lat);
> 
>  	call_object(tables->sample_handler, t, "sample_table");
> 
> --
> 2.25.1