linux-kernel - Re: [PATCH v1 0/8] Run tests in parallel showing number of tests running

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <231a275e-00db-499b-92a4-259dab8f9d80@linaro.org>
Date: Fri, 11 Oct 2024 11:03:03 +0100
From: James Clark <james.clark@...aro.org>
To: Ian Rogers <irogers@...gle.com>
Cc: Peter Zijlstra <peterz@...radead.org>, Ingo Molnar <mingo@...hat.com>,
 Arnaldo Carvalho de Melo <acme@...nel.org>,
 Namhyung Kim <namhyung@...nel.org>, Mark Rutland <mark.rutland@....com>,
 Alexander Shishkin <alexander.shishkin@...ux.intel.com>,
 Jiri Olsa <jolsa@...nel.org>, Adrian Hunter <adrian.hunter@...el.com>,
 Kan Liang <kan.liang@...ux.intel.com>, Howard Chu <howardchu95@...il.com>,
 Athira Jajeev <atrajeev@...ux.vnet.ibm.com>,
 Michael Petlan <mpetlan@...hat.com>, Veronika Molnarova
 <vmolnaro@...hat.com>, Dapeng Mi <dapeng1.mi@...ux.intel.com>,
 Thomas Richter <tmricht@...ux.ibm.com>, Ilya Leoshkevich
 <iii@...ux.ibm.com>, Colin Ian King <colin.i.king@...il.com>,
 Weilin Wang <weilin.wang@...el.com>, Andi Kleen <ak@...ux.intel.com>,
 linux-kernel@...r.kernel.org, linux-perf-users@...r.kernel.org
Subject: Re: [PATCH v1 0/8] Run tests in parallel showing number of tests
 running



On 11/10/2024 8:35 am, Ian Rogers wrote:
> Avoid waitpid so that stdout/stderr aren't destroyed prior to wanting
> to read them for display. When running on a color terminal, display
> the number of running tests (1 if sequential). To avoid previous
> flicker, only delete and refresh the display line when it changes. An
> earlier version of this code is here:
> https://lore.kernel.org/lkml/20240701044236.475098-1-irogers@google.com/
> 
> Add a signal handler for perf tests so that unexpected signals are
> displayed and test clean up is possible.
> 
> In perf test add an "exclusive" flag that causes a test to be run with
> no other test. Set this flag manually for C tests and via a
> "(exclusive)" in the test description for shell tests. Add the flag to
> shell tests that may fail when run with other tests.
> 
> Change the perf test loop to run in two passes. For parallel
> execution, the first pass runs all tests that can be run in parallel
> then the 2nd runs remaining tests sequentially. This causes the
> "exclusive" tests to be run last and with test numbers moderately out
> of alignment.
> 
> Change the default to be to run tests in parallel. Running tests in
> parallel brings the execution time down to less than half.
> 
> Ian Rogers (8):
>    tools subcmd: Add non-waitpid check_if_command_finished()
>    perf test: Display number of remaining tests
>    perf test: Reduce scope of parallel variable
>    perf test: Avoid list test blocking on writing to stdout
>    perf test: Tag parallel failing shell tests with "(exclusive)"
>    perf test: Add a signal handler around running a test
>    perf test: Run parallel tests in two passes
>    perf test: Make parallel testing the default
> 
>   tools/lib/subcmd/run-command.c                |  33 +++
>   tools/perf/tests/builtin-test.c               | 267 ++++++++++++------
>   tools/perf/tests/shell/list.sh                |   5 +-
>   .../tests/shell/perftool-testsuite_report.sh  |   2 +-
>   tools/perf/tests/shell/record.sh              |   2 +-
>   tools/perf/tests/shell/record_lbr.sh          |   2 +-
>   tools/perf/tests/shell/record_offcpu.sh       |   2 +-
>   tools/perf/tests/shell/stat_all_pmu.sh        |   2 +-
>   tools/perf/tests/shell/test_intel_pt.sh       |   2 +-
>   .../perf/tests/shell/test_stat_intel_tpebs.sh |   2 +-
>   tools/perf/tests/tests-scripts.c              |   5 +
>   tools/perf/tests/tests.h                      |   1 +
>   tools/perf/util/color.h                       |   1 +
>   13 files changed, 226 insertions(+), 100 deletions(-)
> 

Not really a big deal but remaining doesn't work when a subset of tests 
are run:

  $ perf test 111 110
  110: Check Arm64 callgraphs are complete in fp mode      : Ok
  111: Check Arm CoreSight trace data recording and synthesized samples:
    Running (150 remaining)

Other than that:

Tested-by: James Clark <james.clark@...aro.org>