linux-kernel - Re: [PATCH] perf test: Update all metrics test like metricgroups test

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAP-5=fXBJ7XpaXJ+9ZChJfkB3QX5jjM20CxvPH14oN=NFjy8eA@mail.gmail.com>
Date: Fri, 18 Oct 2024 14:26:34 -0700
From: Ian Rogers <irogers@...gle.com>
To: Namhyung Kim <namhyung@...nel.org>
Cc: Arnaldo Carvalho de Melo <acme@...nel.org>, Kan Liang <kan.liang@...ux.intel.com>, 
	Jiri Olsa <jolsa@...nel.org>, Adrian Hunter <adrian.hunter@...el.com>, 
	Peter Zijlstra <peterz@...radead.org>, Ingo Molnar <mingo@...nel.org>, 
	LKML <linux-kernel@...r.kernel.org>, linux-perf-users@...r.kernel.org
Subject: Re: [PATCH] perf test: Update all metrics test like metricgroups test

On Fri, Oct 18, 2024 at 1:43 PM Namhyung Kim <namhyung@...nel.org> wrote:
>
> Like in the metricgroup tests, it should check the permission first and
> then skip relevant failures accordingly.
>
> Also it needs to try again with the system wide flag properly.  On the
> second round, check if the result has the metric name because other
> failure cases are checked in the first round already.
>
> Signed-off-by: Namhyung Kim <namhyung@...nel.org>

Reviewed-by: Ian Rogers <irogers@...gle.com>

Thanks,
Ian

> ---
>  tools/perf/tests/shell/stat_all_metrics.sh | 87 ++++++++++++++++------
>  1 file changed, 66 insertions(+), 21 deletions(-)
>
> diff --git a/tools/perf/tests/shell/stat_all_metrics.sh b/tools/perf/tests/shell/stat_all_metrics.sh
> index 54774525e18a7b9a..73e9347e88a964bd 100755
> --- a/tools/perf/tests/shell/stat_all_metrics.sh
> +++ b/tools/perf/tests/shell/stat_all_metrics.sh
> @@ -2,42 +2,87 @@
>  # perf all metrics test
>  # SPDX-License-Identifier: GPL-2.0
>
> +ParanoidAndNotRoot()
> +{
> +  [ "$(id -u)" != 0 ] && [ "$(cat /proc/sys/kernel/perf_event_paranoid)" -gt $1 ]
> +}
> +
> +system_wide_flag="-a"
> +if ParanoidAndNotRoot 0
> +then
> +  system_wide_flag=""
> +fi
> +
>  err=0
>  for m in $(perf list --raw-dump metrics); do
>    echo "Testing $m"
> -  result=$(perf stat -M "$m" true 2>&1)
> -  if [[ "$result" =~ ${m:0:50} ]] || [[ "$result" =~ "<not supported>" ]]
> +  result=$(perf stat -M "$m" $system_wide_flag -- sleep 0.01 2>&1)
> +  result_err=$?
> +  if [[ $result_err -gt 0 ]]
>    then
> -    continue
> +    if [[ "$result" =~ \
> +          "Access to performance monitoring and observability operations is limited" ]]
> +    then
> +      echo "Permission failure"
> +      echo $result
> +      if [[ $err -eq 0 ]]
> +      then
> +        err=2 # Skip
> +      fi
> +      continue
> +    elif [[ "$result" =~ "in per-thread mode, enable system wide" ]]
> +    then
> +      echo "Permissions - need system wide mode"
> +      echo $result
> +      if [[ $err -eq 0 ]]
> +      then
> +        err=2 # Skip
> +      fi
> +      continue
> +    elif [[ "$result" =~ "<not supported>" ]]
> +    then
> +      echo "Not supported events"
> +      echo $result
> +      if [[ $err -eq 0 ]]
> +      then
> +        err=2 # Skip
> +      fi
> +      continue
> +    elif [[ "$result" =~ "FP_ARITH" || "$result" =~ "AMX" ]]
> +    then
> +      echo "FP issues"
> +      echo $result
> +      if [[ $err -eq 0 ]]
> +      then
> +        err=2 # Skip
> +      fi
> +      continue
> +    elif [[ "$result" =~ "PMM" ]]
> +    then
> +      echo "Optane memory issues"
> +      echo $result
> +      if [[ $err -eq 0 ]]
> +      then
> +        err=2 # Skip
> +      fi
> +      continue
> +    fi
>    fi
> -  # Failed so try system wide.
> -  result=$(perf stat -M "$m" -a sleep 0.01 2>&1)
> +
>    if [[ "$result" =~ ${m:0:50} ]]
>    then
>      continue
>    fi
> -  # Failed again, possibly the workload was too small so retry with something
> -  # longer.
> -  result=$(perf stat -M "$m" perf bench internals synthesize 2>&1)
> +
> +  # Failed, possibly the workload was too small so retry with something longer.
> +  result=$(perf stat -M "$m" $system_wide_flag -- perf bench internals synthesize 2>&1)
>    if [[ "$result" =~ ${m:0:50} ]]
>    then
>      continue
>    fi
>    echo "Metric '$m' not printed in:"
>    echo "$result"
> -  if [[ "$err" != "1" ]]
> -  then
> -    err=2
> -    if [[ "$result" =~ "FP_ARITH" || "$result" =~ "AMX" ]]
> -    then
> -      echo "Skip, not fail, for FP issues"
> -    elif [[ "$result" =~ "PMM" ]]
> -    then
> -      echo "Skip, not fail, for Optane memory issues"
> -    else
> -      err=1
> -    fi
> -  fi
> +  err=1
>  done
>
>  exit "$err"
> --
> 2.47.0.rc1.288.g06298d1525-goog
>