linux-kernel - Re: [PATCH v4 0/10] perf: enable compression of record mode trace to save storage space

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20190313143704.GC6676@krava>
Date:   Wed, 13 Mar 2019 15:37:04 +0100
From:   Jiri Olsa <jolsa@...hat.com>
To:     Alexey Budankov <alexey.budankov@...ux.intel.com>
Cc:     Arnaldo Carvalho de Melo <acme@...nel.org>,
        Namhyung Kim <namhyung@...nel.org>,
        Alexander Shishkin <alexander.shishkin@...ux.intel.com>,
        Peter Zijlstra <peterz@...radead.org>,
        Ingo Molnar <mingo@...hat.com>,
        Andi Kleen <ak@...ux.intel.com>,
        linux-kernel <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH v4 0/10] perf: enable compression of record mode trace to
 save storage space

On Thu, Feb 28, 2019 at 11:35:24AM +0300, Alexey Budankov wrote:
> 
> The patch set implements runtime trace compression (-z option) in 
> record mode and trace auto decompression in report and inject modes. 
> Streaming Zstandard (Zstd) API (zstd) is used for compression and 
> decompression of data that come from kernel mmaped data buffers.
> 
> Usage of implemented -z,--compression_level=n option provides ~3-5x 
> avg. trace file size reduction on variety of tested workloads what 
> saves storage space on larger server systems where trace file size 
> can easily reach several tens or even hundreds of GiBs, especially 
> when profiling with dwarf-based stacks and tracing of context switches.
> Implemented -f,--mmap-flush option can be used to avoid compressing 
> every single byte of data and increase compression ratio at the same 
> time lowering tool runtime overhead. Default option value is 1 what 
> is equal to the current perf record implementation. The option is 
> independent from -z setting and doesn't vary with compression level:
> 
>   $ tools/perf/perf record -z 1 -e cycles -- matrix.gcc
>   $ tools/perf/perf record --aio=1 -z 1 -e cycles -- matrix.gcc
>   $ tools/perf/perf record -z 1 -f 1024 -e cycles -- matrix.gcc
>   $ tools/perf/perf record --aio=1 -z 1 -f 1024 -e cycles -- matrix.gcc

please make the 'z' option to choose some default value,
so it's possible just to run:

  perf record -z ...

to get comrpessed data.. not sure which strategy to use,
probably the fastest '-z 1' as default

with some warning about using that default value perhaps

jirka