linux-kernel - Re: [PATCH 7/9] scripts: python: implement get or create frame function

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CAP-5=fUhC7NOxvE=gJFVFU3HtsMe2Jr2uR8ftmozaGtbWn0Ajg@mail.gmail.com>
Date:   Thu, 6 Jul 2023 08:57:57 -0700
From:   Ian Rogers <irogers@...gle.com>
To:     Anup Sharma <anupnewsmail@...il.com>
Cc:     Namhyung Kim <namhyung@...nel.org>,
        Peter Zijlstra <peterz@...radead.org>,
        Ingo Molnar <mingo@...hat.com>,
        Arnaldo Carvalho de Melo <acme@...nel.org>,
        Mark Rutland <mark.rutland@....com>,
        Alexander Shishkin <alexander.shishkin@...ux.intel.com>,
        Jiri Olsa <jolsa@...nel.org>,
        Adrian Hunter <adrian.hunter@...el.com>,
        linux-perf-users@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH 7/9] scripts: python: implement get or create frame function

On Wed, Jul 5, 2023 at 1:01 PM Anup Sharma <anupnewsmail@...il.com> wrote:
>
> On Fri, Jun 23, 2023 at 05:04:56PM -0700, Namhyung Kim wrote:
> > On Wed, Jun 21, 2023 at 12:45 PM Anup Sharma <anupnewsmail@...il.com> wrote:
> > >
> > > The CATEGORIES list and the USER_CATEGORY_INDEX and
> > > KERNEL_CATEGORY_INDEX constants has been introduced.
> > >
> > > The get_or_create_frame function is responsible for retrieving or
> > > creating a frame based on the provided frameString. If the frame
> > > corresponding to the frameString is found in the frameMap, it is
> > > returned. Otherwise, a new frame is created by appending relevant
> > > information to the frameTable's 'data' array and adding the
> > > frameString to the stringTable.
> > >
> > > The index of the newly created frame is added to the frameMap.
> > >
> > > Signed-off-by: Anup Sharma <anupnewsmail@...il.com>
> > > ---
> > >  .../scripts/python/firefox-gecko-converter.py | 38 +++++++++++++++++++
> > >  1 file changed, 38 insertions(+)
> > >
> > > diff --git a/tools/perf/scripts/python/firefox-gecko-converter.py b/tools/perf/scripts/python/firefox-gecko-converter.py
> > > index 30fc542cfdeb..866751e5d1ce 100644
> > > --- a/tools/perf/scripts/python/firefox-gecko-converter.py
> > > +++ b/tools/perf/scripts/python/firefox-gecko-converter.py
> > > @@ -15,6 +15,13 @@ def isPerfScriptFormat(profile):
> > >      firstLine = profile[:profile.index('\n')]
> > >      return bool(re.match(r'^\S.*?\s+(?:\d+/)?\d+\s+(?:\d+\d+\s+)?[\d.]+:', firstLine))
> > >
> > > +CATEGORIES = [
> > > +{'name': 'User', 'color': 'yellow', 'subcategories': ['Other']},
> > > +{'name': 'Kernel', 'color': 'orange', 'subcategories': ['Other']}
> > > +]
> > > +USER_CATEGORY_INDEX = 0
> > > +KERNEL_CATEGORY_INDEX = 1
> > > +
> > >  def convertPerfScriptProfile(profile):
> > >      def _createtread(name, pid, tid):
> > >          markers = {
> > > @@ -70,6 +77,37 @@ def convertPerfScriptProfile(profile):
> > >                  stackMap[key] = stack
> > >              return stack
> > >
> > > +        frameMap = dict()
> > > +        def get_or_create_frame(frameString):
> > > +            frame = frameMap.get(frameString)
> > > +            if frame is None:
> > > +                frame = len(frameTable['data'])
> > > +                location = len(stringTable)
> > > +                stringTable.append(frameString)
> > > +
> > > +                category = KERNEL_CATEGORY_INDEX if frameString.find('kallsyms') != -1 or frameString.find('/vmlinux') != -1 or frameString.endswith('.ko)') else USER_CATEGORY_INDEX
> >
> > This line is too long, we usually don't allow long lines
> > over 100 characters.
>
> Thanks for your suggestion. I have taken care in latest version.
> Is there any way to add such checks in editor itself ? I used checkpatch.pl
> scripts, however it didnt catch this.

Unfortunately checkpatch.pl doesn't work for python code yet. I think
using mypy types would be useful:
https://github.com/python/mypy
Also having docstring on functions would be useful. Some of the code
has some fairly complex indirection and it'd be nice to understand
why.

Thanks,
Ian

> > Thanks,
> > Namhyung
> >
> >
> > > +                implementation = None
> > > +                optimizations = None
> > > +                line = None
> > > +                relevantForJS = False
> > > +                subcategory = None
> > > +                innerWindowID = 0
> > > +                column = None
> > > +
> > > +                frameTable['data'].append([
> > > +                    location,
> > > +                    relevantForJS,
> > > +                    innerWindowID,
> > > +                    implementation,
> > > +                    optimizations,
> > > +                    line,
> > > +                    column,
> > > +                    category,
> > > +                    subcategory,
> > > +                ])
> > > +                frameMap[frameString] = frame
> > > +            return frame
> > > +
> > >          def addSample(threadName, stackArray, time):
> > >              nonlocal name
> > >              if name != threadName:
> > > --
> > > 2.34.1
> > >