linux-kernel - Re: [PATCH v3 4/6] scripts: python: Add trace end processing and JSON output

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Mon, 17 Jul 2023 21:23:07 +0530
From:   Anup Sharma <anupnewsmail@...il.com>
To:     Namhyung Kim <namhyung@...nel.org>,
        Adrian Hunter <adrian.hunter@...el.com>
Cc:     Ian Rogers <irogers@...gle.com>,
        Anup Sharma <anupnewsmail@...il.com>,
        Peter Zijlstra <peterz@...radead.org>,
        Ingo Molnar <mingo@...hat.com>,
        Arnaldo Carvalho de Melo <acme@...nel.org>,
        Mark Rutland <mark.rutland@....com>,
        Alexander Shishkin <alexander.shishkin@...ux.intel.com>,
        Jiri Olsa <jolsa@...nel.org>, linux-perf-users@...r.kernel.org,
        linux-kernel@...r.kernel.org
Subject: Re: [PATCH v3 4/6] scripts: python: Add trace end processing and
 JSON output

On Thu, Jul 13, 2023 at 07:31:36PM -0700, Namhyung Kim wrote:
> Hi Anup and Ian,
> 
> On Wed, Jul 12, 2023 at 10:28 AM Ian Rogers <irogers@...gle.com> wrote:
> >
> > On Mon, Jul 10, 2023 at 4:13 PM Anup Sharma <anupnewsmail@...il.com> wrote:
> > >
> > > Inside the trace end function the final output will be dumped
> > > to standard output in JSON gecko format. Additionally, constants
> > > such as USER_CATEGORY_INDEX, KERNEL_CATEGORY_INDEX, CATEGORIES, and
> > > PRODUCT are defined to provide contextual information.
> > >
> > > Signed-off-by: Anup Sharma <anupnewsmail@...il.com>
> >
> > Acked-by: Ian Rogers <irogers@...gle.com>
> 
> I'm ok with this change too but I think it can be squashed to
> patch 1/6 as I think it'd make it more self-contained.  Of course
> you might change time and thread to have empty values.
> 
> >
> > > ---
> > >  .../scripts/python/firefox-gecko-converter.py | 34 ++++++++++++++++++-
> > >  1 file changed, 33 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/tools/perf/scripts/python/firefox-gecko-converter.py b/tools/perf/scripts/python/firefox-gecko-converter.py
> > > index 0b8a86bdcab1..39818a603265 100644
> > > --- a/tools/perf/scripts/python/firefox-gecko-converter.py
> > > +++ b/tools/perf/scripts/python/firefox-gecko-converter.py
> > > @@ -24,8 +24,40 @@ from Core import *
> > >  thread_map = {}
> > >  start_time = None
> > >
> > > +# Follow Brendan Gregg's Flamegraph convention: orange for kernel and yellow for user
> > > +CATEGORIES = [
> > > +    {'name': 'User', 'color': 'yellow', 'subcategories': ['Other']},
> > > +    {'name': 'Kernel', 'color': 'orange', 'subcategories': ['Other']}
> > > +]
> >
> > A follow up could be to make these command line options, defaulting to
> > orange and yellow.
> 
> Sounds good.

Nice Idea, I have added this in the next version of patch.

> >
> > > +
> > > +# The product name is used by the profiler UI to show the Operating system and Processor.
> > > +PRODUCT = os.popen('uname -op').read().strip()
> 
> I'm not against this but having a command name (or full
> command line) of the target process as a title might be better.
> But I'm not sure if the python scripting engine exposed the info
> (like in perf report --header-only) to the script.

I tried to get the command name or any sort of header information but it seems to
be not exposed to the perf-script-python interface. can anyone confirm
if there is any way to get the command name or any header information from the
perf-script-python interface?

> Thanks,
> Namhyung
> 
> 
> > > +
> > >  def trace_end():
> > > -       pass
> > > +    thread_array = thread_map.values()))
> > > +
> > > +    result = {
> > > +        'meta': {
> > > +            'interval': 1,
> > > +            'processType': 0,
> > > +            'product': PRODUCT,
> > > +            'stackwalk': 1,
> > > +            'debug': 0,
> > > +            'gcpoison': 0,
> > > +            'asyncstack': 1,
> > > +            'startTime': start_time,
> > > +            'shutdownTime': None,
> > > +            'version': 24,
> > > +            'presymbolicated': True,
> > > +            'categories': CATEGORIES,
> > > +            'markerSchema': []
> > > +            },
> > > +        'libs': [],
> > > +        'threads': thread_array,
> > > +        'processes': [],
> > > +        'pausedRanges': []
> > > +    }
> > > +    json.dump(result, sys.stdout, indent=2)
> > >
> > >  def process_event(param_dict):
> > >         global start_time
> > > --
> > > 2.34.1
> > >