linux-kernel - Re: [PATCH v2] perf scripts python: Add Python 3 support to check-perf-trace.py

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Thu, 17 Jan 2019 18:53:36 +0530
From:   Ravi Bangoria <ravi.bangoria@...ux.ibm.com>
To:     Jiri Olsa <jolsa@...hat.com>,
        Seeteena Thoufeek <s1seetee@...ux.vnet.ibm.com>
Cc:     peterz@...radead.org, mingo@...hat.com, acme@...nel.org,
        alexander.shishkin@...ux.intel.com, namhyung@...nel.org,
        linux-kernel@...r.kernel.org,
        Ravi Bangoria <ravi.bangoria@...ux.ibm.com>
Subject: Re: [PATCH v2] perf scripts python: Add Python 3 support to
 check-perf-trace.py

Hi Seeteena, Jiri,

On 1/17/19 6:02 PM, Jiri Olsa wrote:
> On Thu, Jan 17, 2019 at 03:15:28PM +0530, Seeteena Thoufeek wrote:
>> Support both Python 2 and Python 3 in check-perf-trace.py.
>> ``print`` is now a function rather than a statement. This should have
>> no functional change.
>>
>> Fix indentation issue, replace spaces with tab
>>
>> Signed-off-by: Seeteena Thoufeek <s1seetee@...ux.vnet.ibm.com>
>> Reviewed-by: Ravi Bangoria <ravi.bangoria@...ux.ibm.com>
> 
> hum, could you please add some info about testing those changes?
> (or even some global into 0/.. patch)
> 
> this is working for me on python2:
> 
> 	[root@...va perf]# perf script rec check-perf-trace
> 	^C
> 	[root@...va perf]# perf script   -s scripts/python/check-perf-trace.py
> 	trace_begin
> 
> 	unhandled events:
> 
> 
> 	event                                          count
> 
> 	----------------------------------------  -----------
> 
> 	raw_syscalls__sys_enter                      3509879
> 
> 
> but fails for python3:
> 
> 	[root@...-x3650m4-01-vm-04 perf]# perf script rec check-perf-trace
> 	^C[ perf record: Woken up 0 times to write data ]
> 	Warning:
> 	1 out of order events recorded.
> 	[ perf record: Captured and wrote 43.132 MB perf.data (490171 samples) ]
> 
> 	[root@...-x3650m4-01-vm-04 perf]# perf script   -s scripts/python/check-perf-trace.py
> 	Traceback (most recent call last):
> 	  File "scripts/python/check-perf-trace.py", line 18, in <module>
> 	    from perf_trace_context import *
> 	ModuleNotFoundError: No module named 'perf_trace_context'
> 	Error running python script scripts/python/check-perf-trace.py

Yes I'm seeing the same error with check-perf-trace.py.

I think the issue happens only when we use 'perf script -s path/to/script.py'
instead of 'perf script report script_name':

Ex:

  # PERF_EXEC_PATH=`pwd` ./perf.python3 script record stackcollapse
    ^C[ perf record: Woken up 1 times to write data ]
    [ perf record: Captured and wrote 0.263 MB perf.data (497 samples) ]

  # PERF_EXEC_PATH=`pwd` ./perf.python3 script report stackcollapse
    migration/21 1
    migration/41 1
    perf 9
    rngd 1
    sshd 1
    swapper 482
    xfsaild/sda5 2

  # PERF_EXEC_PATH=`pwd` ./perf.python3 script -s scripts/python/stackcollapse.py 
    Traceback (most recent call last):
      File "scripts/python/stackcollapse.py", line 31, in <module>
        from perf_trace_context import *
    ModuleNotFoundError: No module named 'perf_trace_context'
    Error running python script scripts/python/stackcollapse.py

So, it seems, this is not the script issue. Something is wrong when we build perf
with python3?


Here is my test summary on Powerpc Fedora 29:

                              Python2           Python3
 check-perf-trace.py          ok                FAIL (No module named 'perf_trace_context')
 event_analyzing_sample.py    ok                ok
 export-to-sqlite.py          ok                ok
 exported-sql-viewer.py       ok                FAIL (details at the end)
 export-to-postgresql.py      FAIL              FAIL
 failed-syscalls-by-pid.py    ok                ok
 futex-contention.py          ok                ok
 intel-pt-events.py           NOT TESTED        NOT TESTED
 mem-phys-addr.py             NOT TESTED        NOT TESTED
 net_dropmonitor.py           ok                ok
 netdev-times.py              ok                ok
 powerpc-hcalls.py            ok                ok
 sctop.py                     ok                ok
 stackcollapse.py             ok                ok
 stat-cpi.py                  FAIL (Seg fault)  FAIL (Seg fault)
 syscall-counts-by-pid.py     ok                ok
 syscall-counts.py            ok                ok


exported-sql-viewer.py FAILURE:
===============================
  # PERF_EXEC_PATH=`pwd` ./perf.python3 script -s ./scripts/python/exported-sql-viewer.py my_test_db
    Traceback (most recent call last):
    File "./scripts/python/exported-sql-viewer.py", line 2614, in <module>
       Main()
    File "./scripts/python/exported-sql-viewer.py", line 2601, in Main
      db, dbname = dbref.Open("main")
    File "./scripts/python/exported-sql-viewer.py", line 2573, in Open
      raise Exception("Failed to open database " + dbname + " error: " + db.lastError().text())
    Exception: Failed to open database my_test_db error: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
    QPSQL: Unable to connect
    Error running python script ./scripts/python/exported-sql-viewer.py

Thanks,
Ravi