linux-kernel - Re: [PATCH v6 2/2] perf test: Json format checking

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <CAP-5=fXX3wAddCw1+e8hsnqOexs20L3=7b1A=izK45bmhnUHWg@mail.gmail.com>
Date:   Fri, 5 Aug 2022 13:05:19 -0700
From:   Ian Rogers <irogers@...gle.com>
To:     Arnaldo Carvalho de Melo <acme@...nel.org>
Cc:     Peter Zijlstra <peterz@...radead.org>,
        Ingo Molnar <mingo@...hat.com>,
        Mark Rutland <mark.rutland@....com>,
        Alexander Shishkin <alexander.shishkin@...ux.intel.com>,
        Jiri Olsa <jolsa@...nel.org>,
        Namhyung Kim <namhyung@...nel.org>,
        Kan Liang <kan.liang@...ux.intel.com>,
        Zhengjun Xing <zhengjun.xing@...ux.intel.com>,
        Sandipan Das <sandipan.das@....com>,
        Claire Jensen <cjense@...gle.com>, Alyssa Ross <hi@...ssa.is>,
        Like Xu <likexu@...cent.com>,
        James Clark <james.clark@....com>,
        Florian Fischer <florian.fischer@...q.space>,
        linux-kernel@...r.kernel.org, linux-perf-users@...r.kernel.org,
        Claire Jensen <clairej735@...il.com>,
        Stephane Eranian <eranian@...gle.com>
Subject: Re: [PATCH v6 2/2] perf test: Json format checking

On Mon, Jul 18, 2022 at 8:49 AM Arnaldo Carvalho de Melo
<acme@...nel.org> wrote:
>
> Em Mon, Jul 18, 2022 at 12:46:10PM -0300, Arnaldo Carvalho de Melo escreveu:
> > Em Mon, Jul 18, 2022 at 12:44:59PM -0300, Arnaldo Carvalho de Melo escreveu:
> > > Em Thu, Jul 07, 2022 at 01:12:13PM -0700, Ian Rogers escreveu:
> > > > From: Claire Jensen <cjense@...gle.com>
> > > >
> > > > Add field checking tests for perf stat JSON output.
> > > > Sanity checks the expected number of fields are present, that the
> > > > expected keys are present and they have the correct values.
> > >
> > > it isn't installing the lib:
> > >
> > > [root@...e ~]# perf test -v json
> > >  91: perf stat JSON output linter                                    :
> > > --- start ---
> > > test child forked, pid 4086678
> > > Checking json output: no args python3: can't open file '/var/home/acme/libexec/perf-core/tests/shell/lib/perf_json_output_lint.py': [Errno 2] No such file or directory
> > > test child finished with -2
> > > ---- end ----
> > > perf stat JSON output linter: Skip
> > > [root@...e ~]#
> > >
> > > I'm trying to fix, but please test it after installing...
> >
> >
> > It should:
> >
> > install-tests: all install-gtk
> >         $(call QUIET_INSTALL, tests) \
> >                 $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests'; \
> >                 $(INSTALL) tests/attr.py '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests'; \
> >                 $(INSTALL) tests/pe-file.exe* '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests'; \
> >                 $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/attr'; \
> >                 $(INSTALL) tests/attr/* '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/attr'; \
> >                 $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell'; \
> >                 $(INSTALL) tests/shell/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell'; \
> >                 $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> >                 $(INSTALL) tests/shell/lib/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'
> >
> > /me checking...
>
> The patch below is needed, but then should we mix .py and .sh in that
> directory?
>
> ⬢[acme@...lbox perf]$ git diff
> diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
> index 8f0b1fb39984fb7b..65e5ba767fd6210e 100644
> --- a/tools/perf/Makefile.perf
> +++ b/tools/perf/Makefile.perf
> @@ -1006,6 +1006,7 @@ install-tests: all install-gtk
>                 $(INSTALL) tests/shell/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell'; \
>                 $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
>                 $(INSTALL) tests/shell/lib/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'
> +               $(INSTALL) tests/shell/lib/*.py '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'
>
>  install-bin: install-tools install-tests install-traceevent-plugins
>
> ⬢[acme@...lbox perf]$
>
> And then, after that I'm getting:
>
> [root@...e ~]# perf test json
>  91: perf stat JSON output linter                                    : FAILED!
> [root@...e ~]# perf test -v json |& tail -20
> {"core" : "S0-D0-C15", "aggregate-number" : 2, "counter-value" : "2675.000000", "unit" : "", "event" : "stalled-cycles-backend", "event-runtime" : 2864158, "pcnt-running" : 100.00, "metric-value" : 0.341252, "metric-unit" : "backend cycles idle"}
>
> {"core" : "S0-D0-C15", "aggregate-number" : 2, "counter-value" : "233533.000000", "unit" : "", "event" : "instructions", "event-runtime" : 2865528, "pcnt-running" : 100.00, "metric-value" : 0.297920, "metric-unit" : "insn per cycle"}
>
> {"core" : "S0-D0-C15", "aggregate-number" : 2, "metric-value" : 0.192975, "metric-unit" : "stalled cycles per insn"}
>
> {"core" : "S0-D0-C15", "aggregate-number" : 2, "counter-value" : "50214.000000", "unit" : "", "event" : "branches", "event-runtime" : 2865638, "pcnt-running" : 100.00, "metric-value" : 17.295742, "metric-unit" : "M/sec"}
>
> {"core" : "S0-D0-C15", "aggregate-number" : 2, "counter-value" : "1513.000000", "unit" : "", "event" : "branch-misses", "event-runtime" : 1564989, "pcnt-running" : 54.00, "metric-value" : 3.013104, "metric-unit" : "of all branches"}
>
> Traceback (most recent call last):
>   File "/var/home/acme/libexec/perf-core/tests/shell/lib/perf_json_output_lint.py", line 92, in <module>
>     check_json_output(expected_items)
>   File "/var/home/acme/libexec/perf-core/tests/shell/lib/perf_json_output_lint.py", line 53, in check_json_output
>     raise RuntimeError(f'wrong number of fields. counted {count} expected {expected_items}'
> RuntimeError: wrong number of fields. counted 3 expected 8 in '{"core" : "S0-D0-C0", "aggregate-number" : 2, "metric-value" : 0.094345, "metric-unit" : "stalled cycles per insn"}
> '
> test child finished with -1
> ---- end ----
> perf stat JSON output linter: FAILED!
> [root@...e ~]#
>
> Can you please check and send a v7?

Done. Sorry for the delay:
https://lore.kernel.org/lkml/20220805200105.2020995-1-irogers@google.com/

Thanks,
Ian

> - Arnaldo