linux-kernel - Re: [RFCv2 00/48] perf tools: Add threads to record command

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20180914094022.GB96351@gmail.com>
Date:   Fri, 14 Sep 2018 11:40:22 +0200
From:   Ingo Molnar <mingo@...nel.org>
To:     Jiri Olsa <jolsa@...hat.com>
Cc:     Namhyung Kim <namhyung@...nel.org>,
        Alexey Budankov <alexey.budankov@...ux.intel.com>,
        Jiri Olsa <jolsa@...nel.org>,
        Arnaldo Carvalho de Melo <acme@...nel.org>,
        lkml <linux-kernel@...r.kernel.org>,
        Alexander Shishkin <alexander.shishkin@...ux.intel.com>,
        Peter Zijlstra <a.p.zijlstra@...llo.nl>,
        Andi Kleen <andi@...stfloor.org>, kernel-team@....com
Subject: Re: [RFCv2 00/48] perf tools: Add threads to record command

* Jiri Olsa <jolsa@...hat.com> wrote:

> On Fri, Sep 14, 2018 at 11:29:10AM +0900, Namhyung Kim wrote:
> > On Thu, Sep 13, 2018 at 07:10:35PM +0300, Alexey Budankov wrote:
> > > Hi,
> > 
> > Hello,
> > 
> > > 
> > > On 13.09.2018 15:54, Jiri Olsa wrote:
> > > > hi,
> > > > sending *RFC* for threads support in perf record command.
> > > > 
> > > > In big picture this patchset adds perf record --threads
> > > > option that allows to create threads in following modes:
> > > > 
> > > > 1) single thread mode (current)
> > > > 
> > > >   $ perf record ...
> > > >   $ perf record --threads=1 ...
> > > > 
> > > >   - all maps are read/stored under process thread
> > > > 
> > > > 2) mode with specific (X) number of threads
> > > > 
> > > >   $ perf record --threads=X ...
> > > > 
> > > >   - maps are spread equaly among threads
> > > > 
> > > > 3) mode that creates thread for every monitored memory map
> > > > 
> > > >   $ perf record --threads ...
> > > > 
> > > >   - which in perf record is equal to number of CPUs, and
> > > >     it pins each thread to its map's cpu:
> > > > 
> > > > 4) TODO - NUMA aware threads/maps separation
> > > >    ...
> > > > 
> > > > The perf.data stays as a single file.
> > 
> > I'm not sure we really need to keep it as a single file.  As it's a
> > kind of big changes, we might consider breaking compatibility and use
> > a directory structure.
> 
> moving the files into the perf.data at the end is actualy
> not a lot code.. and I think it's one of the 'small' things
> that make this feature more user friendly

So the user shouldn't really care about the structure of the file when most uses of perf 
tooling, and 'single file' versus 'single directory' has similar usability IMHO.

When moving across machines it's recommended to use 'perf archive' anyway, which already 
creates a tarball that includes debuginfo and other context.

In fact keeping the files separate has scalability advantages for 'perf report' and similar 
parsing tools: they could read all the streams in a per-CPU fashion already, from the very 
beginning.

BTW., random annoyance bugreport, for me 'perf archive' is spewing a ton of these messages:

  $ perf archive
  unwind: target platform=x86 is not supported
  unwind: target platform=x86 is not supported
  unwind: target platform=x86 is not supported
  ...
  unwind: target platform=x86 is not supported
  unwind: target platform=x86 is not supported

  Now please run:

  $ tar xvf perf.data.tar.bz2 -C ~/.debug

  wherever you need to run 'perf report' on.

  $ perf version
  perf version 4.19.rc2.gcb48b6

That message is repeated 7,200 times (!) and immediately nuked my terminal history. :-/

Even if we want to emit that warning (we really shouldn't unless it's important for the user to 
know), there's no reason to print thousands of messages to stderr.

Thanks,

	Ingo