linux-kernel - Re: [PATCH v4 0/10] perf: enable compression of record mode trace to save storage space

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <34070359-1a25-b64d-519e-2025185b4b68@linux.intel.com>
Date:   Wed, 13 Mar 2019 18:00:00 +0300
From:   Alexey Budankov <alexey.budankov@...ux.intel.com>
To:     Jiri Olsa <jolsa@...hat.com>
Cc:     Arnaldo Carvalho de Melo <acme@...nel.org>,
        Namhyung Kim <namhyung@...nel.org>,
        Alexander Shishkin <alexander.shishkin@...ux.intel.com>,
        Peter Zijlstra <peterz@...radead.org>,
        Ingo Molnar <mingo@...hat.com>,
        Andi Kleen <ak@...ux.intel.com>,
        linux-kernel <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH v4 0/10] perf: enable compression of record mode trace to
 save storage space


On 13.03.2019 17:37, Jiri Olsa wrote:
> On Thu, Feb 28, 2019 at 11:35:24AM +0300, Alexey Budankov wrote:
>>
>> The patch set implements runtime trace compression (-z option) in 
>> record mode and trace auto decompression in report and inject modes. 
>> Streaming Zstandard (Zstd) API (zstd) is used for compression and 
>> decompression of data that come from kernel mmaped data buffers.
>>
>> Usage of implemented -z,--compression_level=n option provides ~3-5x 
>> avg. trace file size reduction on variety of tested workloads what 
>> saves storage space on larger server systems where trace file size 
>> can easily reach several tens or even hundreds of GiBs, especially 
>> when profiling with dwarf-based stacks and tracing of context switches.
>> Implemented -f,--mmap-flush option can be used to avoid compressing 
>> every single byte of data and increase compression ratio at the same 
>> time lowering tool runtime overhead. Default option value is 1 what 
>> is equal to the current perf record implementation. The option is 
>> independent from -z setting and doesn't vary with compression level:
>>
>>   $ tools/perf/perf record -z 1 -e cycles -- matrix.gcc
>>   $ tools/perf/perf record --aio=1 -z 1 -e cycles -- matrix.gcc
>>   $ tools/perf/perf record -z 1 -f 1024 -e cycles -- matrix.gcc
>>   $ tools/perf/perf record --aio=1 -z 1 -f 1024 -e cycles -- matrix.gcc
> 
> please make the 'z' option to choose some default value,
> so it's possible just to run:
> 
>   perf record -z ...
> 
> to get comrpessed data.. not sure which strategy to use,
> probably the fastest '-z 1' as default
> 
> with some warning about using that default value perhaps

in v8.

~Alexey

> 
> jirka
>