linux-kernel - Re: [PATCH v3 1/3] perf test: Add metric value validation test

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <bc18bc42-3df8-30c9-10ff-8a4ba3d3d9f6@amd.com>
Date:   Fri, 16 Jun 2023 12:32:40 +0530
From:   Ravi Bangoria <ravi.bangoria@....com>
To:     Weilin Wang <weilin.wang@...el.com>
Cc:     Kan Liang <kan.liang@...ux.intel.com>,
        Samantha Alt <samantha.alt@...el.com>,
        Perry Taylor <perry.taylor@...el.com>,
        Caleb Biggers <caleb.biggers@...el.com>,
        Peter Zijlstra <peterz@...radead.org>,
        Ingo Molnar <mingo@...hat.com>,
        Arnaldo Carvalho de Melo <acme@...nel.org>,
        Jiri Olsa <jolsa@...nel.org>,
        Namhyung Kim <namhyung@...nel.org>,
        Adrian Hunter <adrian.hunter@...el.com>,
        Ian Rogers <irogers@...gle.com>,
        linux-perf-users@...r.kernel.org, linux-kernel@...r.kernel.org,
        Ravi Bangoria <ravi.bangoria@....com>
Subject: Re: [PATCH v3 1/3] perf test: Add metric value validation test

Hi,

> diff --git a/tools/perf/tests/shell/stat_metrics_values.sh b/tools/perf/tests/shell/stat_metrics_values.sh
> new file mode 100755
> index 000000000000..65a15c65eea7
> --- /dev/null
> +++ b/tools/perf/tests/shell/stat_metrics_values.sh
> @@ -0,0 +1,30 @@
> +#!/bin/bash
> +# perf metrics value validation
> +# SPDX-License-Identifier: GPL-2.0
> +if [ "x$PYTHON" == "x" ]
> +then
> +	if which python3 > /dev/null
> +	then
> +		PYTHON=python3
> +	else
> +		echo Skipping test, python3 not detected please set environment variable PYTHON.
> +		exit 2
> +	fi
> +fi
> +
> +grep -q Intel /proc/cpuinfo || (echo Skipping non-Intel; exit 2)

This check doesn't seem to be working. On my Zen3 AMD machine:

  $ sudo ./perf test -vvv 105
  105: perf metrics value validation                                   :
  --- start ---
  test child forked, pid 518035
  Skipping non-Intel
  Launch python validation script ./tests/shell/lib/perf_metric_validation.py
  Output will be stored in: /tmp/__perf_test.program.tnPoW
  Starting perf collection
  ...

Interestingly, it passes :)

  ...
  Test validation finished. Final report:
  [
      {
          "Workload": "perf bench futex hash -r 2 -s",
          "Report": {
              "Metric Validation Statistics": {
                  "Total Rule Count": 2,
                  "Passed Rule Count": 2
              },
              "Tests in Category": {
                  "PositiveValueTest": {
                      "Total Tests": 12,
                      "Passed Tests": 12,
                      "Failed Tests": []
                  },
                  "RelationshipTest": {
                      "Total Tests": 0,
                      "Passed Tests": 0,
                      "Failed Tests": []
                  },
                  "SingleMetricTest": {
                      "Total Tests": 2,
                      "Passed Tests": 2,
                      "Failed Tests": []
                  }
              },
              "Errors": []
          }
      }
  ]
  test child finished with 0
  ---- end ----
  perf metrics value validation: Ok

I haven't yet investigated whether it's genuine or false positive.

Thanks,
Ravi