linux-kernel - Re: [External] : Re: RFC Re: [PATCH v2] perf jevents: Parse metrics during conversion

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <259f8e47-bd58-c8dc-c3fd-12e8dd5d013d@oracle.com>
Date:   Thu, 17 Nov 2022 18:16:33 +0000
From:   John Garry <john.g.garry@...cle.com>
To:     Ian Rogers <irogers@...gle.com>,
        Arnaldo Carvalho de Melo <acme@...nel.org>
Cc:     Thomas Richter <tmricht@...ux.ibm.com>,
        Jiri Olsa <jolsa@...nel.org>, Leo Yan <leo.yan@...aro.org>,
        Peter Zijlstra <peterz@...radead.org>,
        Ingo Molnar <mingo@...hat.com>,
        Mark Rutland <mark.rutland@....com>,
        Alexander Shishkin <alexander.shishkin@...ux.intel.com>,
        Namhyung Kim <namhyung@...nel.org>,
        Sumanth Korikkar <sumanthk@...ux.ibm.com>,
        linux-kernel@...r.kernel.org, linux-perf-users@...r.kernel.org
Subject: Re: [External] : Re: RFC Re: [PATCH v2] perf jevents: Parse metrics
 during conversion

On 15/11/2022 17:35, John Garry wrote:
> On 15/11/2022 17:26, Ian Rogers wrote:
>> On Thu, Nov 10, 2022 at 10:48 AM Arnaldo Carvalho de Melo
>> <acme@...nel.org> wrote:
>>>
>>> Em Fri, Oct 28, 2022 at 10:57:02AM -0700, Ian Rogers escreveu:
>>>> On Wed, Oct 12, 2022 at 9:42 AM Ian Rogers <irogers@...gle.com> wrote:
>>>>>
>>>>> Currently the 'MetricExpr' json value is passed from the json
>>>>> file to the pmu-events.c. This change introduces an expression
>>>>> tree that is parsed into. The parsing is done largely by using
>>>>> operator overloading and python's 'eval' function. Two advantages
>>>>> in doing this are:
>>>>>
>>>>> 1) Broken metrics fail at compile time rather than relying on
>>>>>     `perf test` to detect. `perf test` remains relevant for checking
>>>>>     event encoding and actual metric use.
>>>>>
>>>>> 2) The conversion to a string from the tree can minimize the
>>>>>     metric's string size, for example, preferring 1e6 over 1000000
>>>>>     and removing unnecessary whitespace. On x86 this reduces the
>>>>>     string size by 2,823bytes (0.06%).
>>>>>
>>>>> In future changes it would be possible to programmatically
>>>>> generate the json expressions (a single line of text and so a
>>>>> pain to write manually) for an architecture using the expression
>>>>> tree. This could avoid copy-pasting metrics for all architecture
>>>>> variants.
>>>>>
>>>>> Signed-off-by: Ian Rogers <irogers@...gle.com>
>>>>
>>>> Ping, PTAL.
>>>
>>> That would be really nice if people working with JSON, metrics, could
>>> take a look at Ian's work and test it with their files, volunteers?
>>
>> Perhaps John could help?
> 
> I'll have a look soon. I have to admit that I have not been tracking the 
> jevents changes as close as before.

At least I can try to test it... so we support python 3.6 and later, right?

I have 3.6 and acme perf/core generated pmu-events.c is ok for me, but 
this patch causes an error:

   GEN     pmu-events/pmu-events.c
  PERF_VERSION = 6.1.rc3.g39b7ecfa5b4a
  GEN     perf-archive
  GEN     perf-iostat
Traceback (most recent call last):
  File "pmu-events/jevents.py", line 7, in <module>
    import metric
  File "/home/john/kernel-dev2/tools/perf/pmu-events/metric.py", line 
399, in <module>
    class MetricGroup:
  File "/home/john/kernel-dev2/tools/perf/pmu-events/metric.py", line 
408, in MetricGroup
    'MetricGroup']]):
TypeError: 'type' object is not subscriptable
make[3]: *** [pmu-events/Build:26: pmu-events/pmu-events.c] Error 1
make[2]: *** [Makefile.perf:672: pmu-events/pmu-events-in.o] Error 2
make[2]: *** Waiting for unfinished jobs....


Any idea?

Thanks,
John