linux-kernel - Re: [PATCH V3] perf scripts python: Add a script to run instances of perf script in parallel

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <ZikT69GIsijZajoI@x1>
Date: Wed, 24 Apr 2024 11:15:07 -0300
From: Arnaldo Carvalho de Melo <acme@...nel.org>
To: Andi Kleen <ak@...ux.intel.com>
Cc: Adrian Hunter <adrian.hunter@...el.com>, Jiri Olsa <jolsa@...nel.org>,
	Namhyung Kim <namhyung@...nel.org>, Ian Rogers <irogers@...gle.com>,
	linux-kernel@...r.kernel.org, linux-perf-users@...r.kernel.org
Subject: Re: [PATCH V3] perf scripts python: Add a script to run instances of
 perf script in parallel

On Tue, Apr 23, 2024 at 04:33:53PM -0700, Andi Kleen wrote:
> On Tue, Apr 23, 2024 at 04:32:48PM +0300, Adrian Hunter wrote:
> > Add a Python script to run a perf script command multiple times in
> > parallel, using perf script options --cpu and --time so that each job
> > processes a different chunk of the data.
> > 
> > Extend perf script tests to test also the new script.
> > 
> > The script supports the use of normal perf script options like
> >  --dlfilter and --script, so that the benefit of running parallel jobs
> > naturally extends to them also. In addition, a command can be provided
> > (refer --pipe-to option) to pipe standard output to a custom command.
> > 
> > Refer to the script's own help text at the end of the patch for more
> > details.
> > 
> > The script is useful for Intel PT traces, that can be efficiently
> > decoded by perf script when split by CPU and/or time ranges. Running
> > jobs in parallel can decrease the overall decoding time.
> > 
> > Signed-off-by: Adrian Hunter <adrian.hunter@...el.com>
> 
> Reviewed-by: Andi Kleen <ak@...ux.intel.com>

Thanks, applied, and added a note on testing it using:

      root@...ber:~# perf test -vv "perf script tests"
       97: perf script tests:
      <SNIP>
      Starting: perf script --cpu=27 -i /tmp/perf-test-script.T4MJDr0L6J/pp-perf.data
      Finished: perf script --cpu=25 -i /tmp/perf-test-script.T4MJDr0L6J/pp-perf.data
      Finished: perf script --cpu=26 -i /tmp/perf-test-script.T4MJDr0L6J/pp-perf.data
      Finished: perf script --cpu=27 -i /tmp/perf-test-script.T4MJDr0L6J/pp-perf.data
      There are 28 jobs: 27 completed, 1 running 
      Finished: perf script --cpu=24 -i /tmp/perf-test-script.T4MJDr0L6J/pp-perf.data
      There are 28 jobs: 28 completed, 0 running 
      All jobs finished successfully                      
      parallel-perf.py done
      parallel-perf test [Success]
      --- Cleaning up ---
      ---- end(0) ----
       97: perf script tests                                               : Ok
      root@...ber:~#

- Arnaldo