linux-kernel - Re: [PATCH v9 12/12] perf tests: implement Zstd comp/decomp integration test

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <a4525d8b-b2f8-35a8-9acb-5278fe83055a@linux.intel.com>
Date:   Mon, 18 Mar 2019 15:07:57 +0300
From:   Alexey Budankov <alexey.budankov@...ux.intel.com>
To:     Jiri Olsa <jolsa@...hat.com>
Cc:     Arnaldo Carvalho de Melo <acme@...nel.org>,
        Namhyung Kim <namhyung@...nel.org>,
        Alexander Shishkin <alexander.shishkin@...ux.intel.com>,
        Ingo Molnar <mingo@...hat.com>,
        Peter Zijlstra <peterz@...radead.org>,
        Andi Kleen <ak@...ux.intel.com>,
        linux-kernel <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH v9 12/12] perf tests: implement Zstd comp/decomp
 integration test

On 18.03.2019 14:46, Jiri Olsa wrote:
> On Sat, Mar 16, 2019 at 05:18:24PM +0300, Alexey Budankov wrote:
>>
>> Implemented basic integration test for Zstd based trace
>> compression/decompression in record and report modes.
>>
>> Signed-off-by: Alexey Budankov <alexey.budankov@...ux.intel.com>
>> ---
>>  .../tests/shell/record+zstd_comp_decomp.sh    | 28 +++++++++++++++++++
>>  1 file changed, 28 insertions(+)
>>  create mode 100755 tools/perf/tests/shell/record+zstd_comp_decomp.sh
>>
>> diff --git a/tools/perf/tests/shell/record+zstd_comp_decomp.sh b/tools/perf/tests/shell/record+zstd_comp_decomp.sh
>> new file mode 100755
>> index 000000000000..bc195874fc27
>> --- /dev/null
>> +++ b/tools/perf/tests/shell/record+zstd_comp_decomp.sh
>> @@ -0,0 +1,28 @@
>> +#!/bin/sh
>> +# record trace Zstd compression/decompression
>> +
>> +trace_file=$(mktemp /tmp/perf.data.XXX)
>> +perf_tool=perf
>> +output=/dev/null
>> +
>> +skip_if_no_z_record() {
>> +	$perf_tool record -h 2>&1 | grep '\-z, \-\-compression\-level'
>> +}
>> +
>> +collect_z_record() {
>> +	echo "Collecting compressed record trace file:"
>> +	$perf_tool record -o $trace_file -a -z -F 25000 -e cycles -- \
>> +		dd count=2000 if=/dev/random of=/dev/null > $output 2>&1
>> +}
>> +
>> +check_compressed_stats() {
>> +	echo "Checking compressed events stats:"
>> +	$perf_tool report -i $trace_file --header --stats | \
>> +		grep -E "(# compressed : Zstd,)|(COMPRESSED events:)" > $output 2>&1
>> +}
>> +
>> +skip_if_no_z_record || exit 2
>> +collect_z_record && check_compressed_stats
>> +err=$?
>> +rm -f $trace_file
>> +exit $err
>> -- 
>> 2.20.1
>>
> 
> could we also check the actualy decompression like below
> 
> plus let's disable -a so non root user can run it,
> and also it takes too long.. changing dd to count=1000
> and adding -g to record

in v10.

~Alexey

> 
> jirka
> 
> 
> ---
> diff --git a/tools/perf/tests/shell/record+zstd_comp_decomp.sh b/tools/perf/tests/shell/record+zstd_comp_decomp.sh
> index bc195874fc27..c0ff142bf66a 100755
> --- a/tools/perf/tests/shell/record+zstd_comp_decomp.sh
> +++ b/tools/perf/tests/shell/record+zstd_comp_decomp.sh
> @@ -11,8 +11,8 @@ skip_if_no_z_record() {
>  
>  collect_z_record() {
>  	echo "Collecting compressed record trace file:"
> -	$perf_tool record -o $trace_file -a -z -F 25000 -e cycles -- \
> -		dd count=2000 if=/dev/random of=/dev/null > $output 2>&1
> +	$perf_tool record -o $trace_file -g -z -F 25000 -- \
> +		dd count=1000 if=/dev/random of=/dev/null > $output 2>&1
>  }
>  
>  check_compressed_stats() {
> @@ -21,8 +21,15 @@ check_compressed_stats() {
>  		grep -E "(# compressed : Zstd,)|(COMPRESSED events:)" > $output 2>&1
>  }
>  
> +check_compressed_output() {
> +	$perf_tool inject -i $trace_file -o $trace_file.decomp &&
> +	$perf_tool report -i $trace_file --stdio | head -n -3 > $trace_file.comp.output &&
> +	$perf_tool report -i $trace_file.decomp --stdio | head -n -3 > $trace_file.decomp.output &&
> +	diff $trace_file.comp.output $trace_file.decomp.output > $output 2>&1
> +}
> +
>  skip_if_no_z_record || exit 2
> -collect_z_record && check_compressed_stats
> +collect_z_record && check_compressed_stats && check_compressed_output
>  err=$?
> -rm -f $trace_file
> +rm -f $trace_file*
>  exit $err
>