linux-kernel - RE: [PATCH v3 1/3] perf test: Add metric value validation test

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CO6PR11MB56353207BBD7BDA6C530EBDEEE5EA@CO6PR11MB5635.namprd11.prod.outlook.com>
Date:   Sun, 18 Jun 2023 17:43:12 +0000
From:   "Wang, Weilin" <weilin.wang@...el.com>
To:     Ravi Bangoria <ravi.bangoria@....com>
CC:     Kan Liang <kan.liang@...ux.intel.com>,
        "Alt, Samantha" <samantha.alt@...el.com>,
        "Taylor, Perry" <perry.taylor@...el.com>,
        "Biggers, Caleb" <caleb.biggers@...el.com>,
        Peter Zijlstra <peterz@...radead.org>,
        Ingo Molnar <mingo@...hat.com>,
        Arnaldo Carvalho de Melo <acme@...nel.org>,
        Jiri Olsa <jolsa@...nel.org>,
        Namhyung Kim <namhyung@...nel.org>,
        "Hunter, Adrian" <adrian.hunter@...el.com>,
        Ian Rogers <irogers@...gle.com>,
        "linux-perf-users@...r.kernel.org" <linux-perf-users@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: RE: [PATCH v3 1/3] perf test: Add metric value validation test



> -----Original Message-----
> From: Ravi Bangoria <ravi.bangoria@....com>
> Sent: Friday, June 16, 2023 12:03 AM
> To: Wang, Weilin <weilin.wang@...el.com>
> Cc: Kan Liang <kan.liang@...ux.intel.com>; Alt, Samantha
> <samantha.alt@...el.com>; Taylor, Perry <perry.taylor@...el.com>; Biggers,
> Caleb <caleb.biggers@...el.com>; Peter Zijlstra <peterz@...radead.org>; Ingo
> Molnar <mingo@...hat.com>; Arnaldo Carvalho de Melo <acme@...nel.org>;
> Jiri Olsa <jolsa@...nel.org>; Namhyung Kim <namhyung@...nel.org>; Hunter,
> Adrian <adrian.hunter@...el.com>; Ian Rogers <irogers@...gle.com>; linux-
> perf-users@...r.kernel.org; linux-kernel@...r.kernel.org; Ravi Bangoria
> <ravi.bangoria@....com>
> Subject: Re: [PATCH v3 1/3] perf test: Add metric value validation test
> 
> Hi,
> 
> > diff --git a/tools/perf/tests/shell/stat_metrics_values.sh
> b/tools/perf/tests/shell/stat_metrics_values.sh
> > new file mode 100755
> > index 000000000000..65a15c65eea7
> > --- /dev/null
> > +++ b/tools/perf/tests/shell/stat_metrics_values.sh
> > @@ -0,0 +1,30 @@
> > +#!/bin/bash
> > +# perf metrics value validation
> > +# SPDX-License-Identifier: GPL-2.0
> > +if [ "x$PYTHON" == "x" ]
> > +then
> > +	if which python3 > /dev/null
> > +	then
> > +		PYTHON=python3
> > +	else
> > +		echo Skipping test, python3 not detected please set
> environment variable PYTHON.
> > +		exit 2
> > +	fi
> > +fi
> > +
> > +grep -q Intel /proc/cpuinfo || (echo Skipping non-Intel; exit 2)
> 
> This check doesn't seem to be working. On my Zen3 AMD machine:

Thanks for reporting this! I've update this search to "GenuineIntel" in v4 to help solve this issue. 
Please check it out. 

> 
>   $ sudo ./perf test -vvv 105
>   105: perf metrics value validation                                   :
>   --- start ---
>   test child forked, pid 518035
>   Skipping non-Intel
>   Launch python validation script ./tests/shell/lib/perf_metric_validation.py
>   Output will be stored in: /tmp/__perf_test.program.tnPoW
>   Starting perf collection
>   ...
> 
> Interestingly, it passes :)
> 
>   ...
>   Test validation finished. Final report:
>   [
>       {
>           "Workload": "perf bench futex hash -r 2 -s",
>           "Report": {
>               "Metric Validation Statistics": {
>                   "Total Rule Count": 2,
>                   "Passed Rule Count": 2
>               },
>               "Tests in Category": {
>                   "PositiveValueTest": {
>                       "Total Tests": 12,
>                       "Passed Tests": 12,
>                       "Failed Tests": []
>                   },
>                   "RelationshipTest": {
>                       "Total Tests": 0,
>                       "Passed Tests": 0,
>                       "Failed Tests": []
>                   },
>                   "SingleMetricTest": {
>                       "Total Tests": 2,
>                       "Passed Tests": 2,
>                       "Failed Tests": []
>                   }
>               },
>               "Errors": []
>           }
>       }
>   ]
>   test child finished with 0
>   ---- end ----
>   perf metrics value validation: Ok
> 
> I haven't yet investigated whether it's genuine or false positive.
> 
Base on this final report, it validated some basic rules like the 12 metrics for positive value test and 2 metrics for single metric value checks. 
The test script grabs metrics supported on the platform and generates validation rules that only include metrics in the supported list. 
Therefore, it is not surprising that the check passes on your system. 

Thanks,
Weilin

> Thanks,
> Ravi