linux-kernel - Re: [PATCH 4/4] perf test: Do not skip when some metric-group tests succeed

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CAP-5=fVO+EK-vaEfvxbmmtReNJJNYpx31W62E_FLBDpMe5a3Lw@mail.gmail.com>
Date: Fri, 9 Jan 2026 15:39:50 -0800
From: Ian Rogers <irogers@...gle.com>
To: Namhyung Kim <namhyung@...nel.org>
Cc: Arnaldo Carvalho de Melo <acme@...nel.org>, James Clark <james.clark@...aro.org>, 
	Jiri Olsa <jolsa@...nel.org>, Adrian Hunter <adrian.hunter@...el.com>, 
	Peter Zijlstra <peterz@...radead.org>, Ingo Molnar <mingo@...nel.org>, 
	LKML <linux-kernel@...r.kernel.org>, linux-perf-users@...r.kernel.org
Subject: Re: [PATCH 4/4] perf test: Do not skip when some metric-group tests succeed

On Thu, Dec 18, 2025 at 5:18 PM Namhyung Kim <namhyung@...nel.org> wrote:
>
> I think the return value of SKIP (2) should be used when it skipped the
> entire test suite rather than a few of them.  While the FAIL should be
> reserved if any of test failed.

The same feedback as the last patch:
https://lore.kernel.org/lkml/CAP-5=fVDYu+GLspYQNEHk7_X8-FE-OaEF0OaG+vfWrrYH9ZcoQ@mail.gmail.com/

Thanks,
Ian

>   $ perf test -vv 109
>   109: perf all metricgroups test:
>   --- start ---
>   test child forked, pid 2493003
>   Testing Backend
>   Testing Bad
>   Testing BadSpec
>   Testing BigFootprint
>   Testing BrMispredicts
>   Testing Branches
>   Testing BvBC
>   Testing BvBO
>   Testing BvCB
>   Testing BvFB
>   Testing BvIO
>   Testing BvMB
>   Testing BvML
>   Testing BvMP
>   Testing BvMS
>   Testing BvMT
>   Testing BvOB
>   Testing BvUW
>   Testing CacheHits
>   Testing CacheMisses
>   Testing CodeGen
>   Testing Compute
>   Testing Cor
>   Testing DSB
>   Testing DSBmiss
>   Testing DataSharing
>   Testing Default
>   Testing Default2
>   Testing Default3
>   Testing Default4
>   Ignoring failures in Default4 that may contain unsupported legacy events
>   Testing Fed
>   Testing FetchBW
>   Testing FetchLat
>   Testing Flops
>   Testing FpScalar
>   Testing FpVector
>   Testing Frontend
>   Testing HPC
>   Testing IcMiss
>   Testing InsType
>   Testing LSD
>   Testing LockCont
>   Testing MachineClears
>   Testing Machine_Clears
>   Testing Mem
>   Testing MemOffcore
>   Testing MemoryBW
>   Testing MemoryBound
>   Testing MemoryLat
>   Testing MemoryTLB
>   Testing Memory_BW
>   Testing Memory_Lat
>   Testing MicroSeq
>   Testing OS
>   Testing Offcore
>   Testing PGO
>   Testing Pipeline
>   Testing PortsUtil
>   Testing Power
>   Testing Prefetches
>   Testing Ret
>   Testing Retire
>   Testing SMT
>   Testing Snoop
>   Testing SoC
>   Testing Summary
>   Testing TmaL1
>   Testing TmaL2
>   Testing TmaL3mem
>   Testing TopdownL1
>   Testing TopdownL2
>   Testing TopdownL3
>   Testing TopdownL4
>   Testing TopdownL5
>   Testing TopdownL6
>   Testing smi
>   Testing tma_L1_group
>   Testing tma_L2_group
>   Testing tma_L3_group
>   Testing tma_L4_group
>   Testing tma_L5_group
>   Testing tma_L6_group
>   Testing tma_alu_op_utilization_group
>   Testing tma_assists_group
>   Testing tma_backend_bound_group
>   Testing tma_bad_speculation_group
>   Testing tma_branch_mispredicts_group
>   Testing tma_branch_resteers_group
>   Testing tma_code_stlb_miss_group
>   Testing tma_core_bound_group
>   Testing tma_divider_group
>   Testing tma_dram_bound_group
>   Testing tma_dtlb_load_group
>   Testing tma_dtlb_store_group
>   Testing tma_fetch_bandwidth_group
>   Testing tma_fetch_latency_group
>   Testing tma_fp_arith_group
>   Testing tma_fp_vector_group
>   Testing tma_frontend_bound_group
>   Testing tma_heavy_operations_group
>   Testing tma_icache_misses_group
>   Testing tma_issue2P
>   Testing tma_issueBM
>   Testing tma_issueBW
>   Testing tma_issueComp
>   Testing tma_issueD0
>   Testing tma_issueFB
>   Testing tma_issueFL
>   Testing tma_issueL1
>   Testing tma_issueLat
>   Testing tma_issueMC
>   Testing tma_issueMS
>   Testing tma_issueMV
>   Testing tma_issueRFO
>   Testing tma_issueSL
>   Testing tma_issueSO
>   Testing tma_issueSmSt
>   Testing tma_issueSpSt
>   Testing tma_issueSyncxn
>   Testing tma_issueTLB
>   Testing tma_itlb_misses_group
>   Testing tma_l1_bound_group
>   Testing tma_l2_bound_group
>   Testing tma_l3_bound_group
>   Testing tma_light_operations_group
>   Testing tma_load_stlb_miss_group
>   Testing tma_machine_clears_group
>   Testing tma_memory_bound_group
>   Testing tma_microcode_sequencer_group
>   Testing tma_mite_group
>   Testing tma_other_light_ops_group
>   Testing tma_ports_utilization_group
>   Testing tma_ports_utilized_0_group
>   Testing tma_ports_utilized_3m_group
>   Testing tma_retiring_group
>   Testing tma_serializing_operation_group
>   Testing tma_store_bound_group
>   Testing tma_store_stlb_miss_group
>   Testing transaction
>   ---- end(0) ----
>   109: perf all metricgroups test                                      : Ok
>
> Signed-off-by: Namhyung Kim <namhyung@...nel.org>
> ---
>  .../perf/tests/shell/stat_all_metricgroups.sh | 26 ++++++++++++-------
>  1 file changed, 16 insertions(+), 10 deletions(-)
>
> diff --git a/tools/perf/tests/shell/stat_all_metricgroups.sh b/tools/perf/tests/shell/stat_all_metricgroups.sh
> index 1400880ec01f8267..81bc7070b5ab0d5c 100755
> --- a/tools/perf/tests/shell/stat_all_metricgroups.sh
> +++ b/tools/perf/tests/shell/stat_all_metricgroups.sh
> @@ -12,31 +12,32 @@ if ParanoidAndNotRoot 0
>  then
>    system_wide_flag=""
>  fi
> -err=0
> +
> +err=3
> +skip=0
>  for m in $(perf list --raw-dump metricgroups)
>  do
>    echo "Testing $m"
>    result=$(perf stat -M "$m" $system_wide_flag sleep 0.01 2>&1)
>    result_err=$?
> -  if [[ $result_err -gt 0 ]]
> +  if [[ $result_err -eq 0 ]]
>    then
> +    if [[ "$err" -ne 1 ]]
> +    then
> +      err=0
> +    fi
> +  else
>      if [[ "$result" =~ \
>            "Access to performance monitoring and observability operations is limited" ]]
>      then
>        echo "Permission failure"
>        echo $result
> -      if [[ $err -eq 0 ]]
> -      then
> -        err=2 # Skip
> -      fi
> +      skip=1
>      elif [[ "$result" =~ "in per-thread mode, enable system wide" ]]
>      then
>        echo "Permissions - need system wide mode"
>        echo $result
> -      if [[ $err -eq 0 ]]
> -      then
> -        err=2 # Skip
> -      fi
> +      skip=1
>      elif [[ "$m" == @(Default2|Default3|Default4) ]]
>      then
>        echo "Ignoring failures in $m that may contain unsupported legacy events"
> @@ -48,4 +49,9 @@ do
>    fi
>  done
>
> +if [[ "$err" -eq 3 && "$skip" -eq 1 ]]
> +then
> +  err=2
> +fi
> +
>  exit $err
> --
> 2.52.0.322.g1dd061c0dc-goog
>