linux-kernel - Re: [PATCH] perf data: Allow to use stdio functions for pipe mode

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAM9d7cgikTxC9s86OYp_owoJ7JpdoC3PC42iC4oQpaxbMeudaw@mail.gmail.com>
Date:   Thu, 29 Oct 2020 20:15:19 +0900
From:   Namhyung Kim <namhyung@...nel.org>
To:     Ian Rogers <irogers@...gle.com>
Cc:     Arnaldo Carvalho de Melo <acme@...nel.org>,
        Jiri Olsa <jolsa@...hat.com>, Ingo Molnar <mingo@...nel.org>,
        Peter Zijlstra <a.p.zijlstra@...llo.nl>,
        Mark Rutland <mark.rutland@....com>,
        Alexander Shishkin <alexander.shishkin@...ux.intel.com>,
        LKML <linux-kernel@...r.kernel.org>,
        Stephane Eranian <eranian@...gle.com>
Subject: Re: [PATCH] perf data: Allow to use stdio functions for pipe mode

Hi Ian,

On Thu, Oct 29, 2020 at 1:37 AM Ian Rogers <irogers@...gle.com> wrote:
>
> On Wed, Oct 28, 2020 at 1:56 AM Namhyung Kim <namhyung@...nel.org> wrote:
> >
> > When perf data is in a pipe, it reads each event separately using
> > read(2) syscall.  This is a huge performance bottleneck when
> > processing large data like in perf inject.  Also perf inject needs to
> > use write(2) syscall for the output.
> >
> > So convert it to use buffer I/O functions in stdio library for pipe
> > data.  This makes inject-build-id bench time drops from 20ms to 8ms.
>
> This is great! Reading the numbers below the times are reduced to
> 67.9% of their previous, this is a little less than 20ms to 8ms so
> perhaps check that number.

The numbers before this patch:

$ ./perf bench internals inject-build-id
# Running 'internals/inject-build-id' benchmark:
  Average build-id injection took: 20.594 msec (+- 0.034 msec)
  Average time per event: 2.019 usec (+- 0.003 usec)
  Average memory usage: 8319 KB (+- 0 KB)
  Average build-id-all injection took: 19.443 msec (+- 0.177 msec)
  Average time per event: 1.906 usec (+- 0.017 usec)
  Average memory usage: 7490 KB (+- 0 KB)

>
> >   $ perf bench internals inject-build-id
> >   # Running 'internals/inject-build-id' benchmark:
> >     Average build-id injection took: 8.074 msec (+- 0.013 msec)
> >     Average time per event: 0.792 usec (+- 0.001 usec)
> >     Average memory usage: 8328 KB (+- 0 KB)
> >     Average build-id-all injection took: 5.490 msec (+- 0.008 msec)
> >     Average time per event: 0.538 usec (+- 0.001 usec)
> >     Average memory usage: 7563 KB (+- 0 KB)
> >
> > This patch enables it just for perf inject when used with pipe (it's a
> > default behavior).  Maybe we could do it for perf record and/or report
> > later..
>
> For perf record there is also the async IO used for writing. I like
> that this patch is adding the reading to perf_data. Should the async
> IO code live in perf_data rather than record? Should async IO be used
> for reading in that case? What would be the policy of using stdio,
> not-stdio and async IO?

I'm not sure but using async IO for read is meaningless since we cannot
do anything without the data.  The reason I added this was that the pipe
mode does so many small I/O with syscalls.  But perf record will mostly
do large I/O so I don't expect big speedup for that.

Thanks
Namhyung