Message-ID: <ZrZ7JvzGo9QRap8K@x1>
Date: Fri, 9 Aug 2024 17:25:10 -0300
From: Arnaldo Carvalho de Melo <acme@...nel.org>
To: Greg Kroah-Hartman <gregkh@...uxfoundation.org>
Cc: "John B. Wyatt IV" <jwyatt@...hat.com>,
	Shuah Khan <skhan@...uxfoundation.org>, linux-pm@...r.kernel.org,
	Thomas Renninger <trenn@...e.com>, Shuah Khan <shuah@...nel.org>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	linux-kernel@...r.kernel.org, John Kacur <jkacur@...hat.com>,
	Tomas Glozar <tglozar@...hat.com>,
	"John B. Wyatt IV" <sageofredondo@...il.com>
Subject: Re: [PATCH 0/2][RFC] Add SWIG Bindings to libcpupower

On Sun, Aug 04, 2024 at 10:54:10AM +0200, Greg Kroah-Hartman wrote:
> On Thu, Aug 01, 2024 at 05:24:20PM -0400, John B. Wyatt IV wrote:
> > > On 7/24/24 16:11, John B. Wyatt IV wrote:
> > > > SWIG is a tool packaged in Fedora and other distros that can generate
> > > > bindings from C and C++ code for several languages including Python,
> > > > Perl, and Go. We at Red Hat are interested in adding binding support to
> > > > libcpupower so Python tools like rteval or tuned can make easy use of it.
> > > > 
> > > 
> > > Can you elaborate on the use-case and what rteval currently does and
> > > how it could benefit from using libcpupower with the bindings?
> > 
> > rteval is a Python program used to measure realtime performance. We wanted to
> > test the effect of enabling some levels of idle-stat to see how it affects
> > latency, and didn't want to reinvent the wheel. We thought that the Python
> > bindings could be useful to other people as well who might want to call
> > cpupower too from Python. I did some testing and was able to achieve this with
> > SWIG. We sent the patchset to see what folks thought about this.
 
> Is this going to require a built-time dependency on SWIG?  If not, when
> would it be run, and who will be in charge of running it and updating
> the bindings?

> And finally, why do we need these at all?  You are saying these are new
> tests that external tools will be using, but why, if external tools are
> required to run them, are they needed in the kernel tree at all?  Why
> isn't this just another external test-suite that people who care about
> measuring this type of thing going to just run on their own if desired?

We have a python binding for perf, it was done manually, didn't use
something like swig, don't recall why I made that decision at the time,
maybe not to get one extra build dependency, as you mention.

I did it at the time as another tool developed at Red Hat, tuna, a top
like tool, using GTK+, could use the perf infrastructure to get
notifications about new threads and threads exiting, as exemplified by
test tools in the kernel sources:

root@...ber:/home/acme/git/linux# tools/perf/python/twatch.py 
cpu: 24, pid: 2787405, tid: 2787405 { type: fork, pid: 2787405, ppid: 2787405, tid: 673409, ptid: 2787405, time: 300460407945015}
cpu: 23, pid: 2787405, tid: 673409 { type: comm, pid: 2787405, tid: 673409, comm: StreamT~s #1624 }
cpu: 4, pid: 3923381, tid: 3923381 { type: fork, pid: 3923381, ppid: 3923381, tid: 673410, ptid: 3923381, time: 300460862312442}
cpu: 23, pid: 2787405, tid: 673409 { type: comm, pid: 2787405, tid: 673409, comm: StreamT~s #1624 }
cpu: 23, pid: 3923381, tid: 673410 { type: comm, pid: 3923381, tid: 673410, comm: StreamT~s #3197 }
cpu: 23, pid: 3923381, tid: 673410 { type: comm, pid: 3923381, tid: 673410, comm: StreamT~s #3197 }
cpu: 13, pid: 2787464, tid: 2787464 { type: fork, pid: 2787464, ppid: 2787464, tid: 673411, ptid: 2787464, time: 300461205531219}
cpu: 26, pid: 2787464, tid: 673411 { type: comm, pid: 2787464, tid: 673411, comm: StreamT~s #1624 }
cpu: 26, pid: 2787464, tid: 673411 { type: comm, pid: 2787464, tid: 673411, comm: StreamT~s #1624 }
^CTraceback (most recent call last):
  File "/home/acme/git/linux/tools/perf/python/twatch.py", line 61, in <module>
    main()
  File "/home/acme/git/linux/tools/perf/python/twatch.py", line 33, in main
    evlist.poll(timeout = -1)
KeyboardInterrupt

root@...ber:/home/acme/git/linux# 

root@...ber:/home/acme/git/linux# head tools/perf/python/twatch.py 
#! /usr/bin/env python
# SPDX-License-Identifier: GPL-2.0-only
# -*- python -*-
# -*- coding: utf-8 -*-
#   twatch - Experimental use of the perf python interface
#   Copyright (C) 2011 Arnaldo Carvalho de Melo <acme@...hat.com>
#

import perf

root@...ber:/home/acme/git/linux#

Another

root@...ber:/home/acme/git/linux# tools/perf/python/tracepoint.py | head -50 | tail
time 302243176736731 prev_comm=chromium-browse prev_pid=633048 prev_prio=120 prev_state=0x1 ==> next_comm=swapper/8 next_pid=0 next_prio=120
time 302243177970076 prev_comm=swapper/10 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=python next_pid=676159 next_prio=120
time 302243177952044 prev_comm=Compositor prev_pid=660727 prev_prio=120 prev_state=0x1 ==> next_comm=migration/17 next_pid=74 next_prio=0
time 302243176742113 prev_comm=swapper/8 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=chromium-browse next_pid=633048 next_prio=120
time 302243177991642 prev_comm=python prev_pid=676159 prev_prio=120 prev_state=0x1 ==> next_comm=swapper/10 next_pid=0 next_prio=120
time 302243177962820 prev_comm=migration/17 prev_pid=74 prev_prio=0 prev_state=0x1 ==> next_comm=swapper/17 next_pid=0 next_prio=120
time 302243176784747 prev_comm=chromium-browse prev_pid=633048 prev_prio=120 prev_state=0x1 ==> next_comm=swapper/8 next_pid=0 next_prio=120
time 302243177993435 prev_comm=swapper/10 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=python next_pid=676159 next_prio=120
time 302243177966620 prev_comm=swapper/17 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=Compositor next_pid=660727 next_prio=120
time 302243176999285 prev_comm=swapper/8 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=chromium-browse next_pid=633048 next_prio=120
root@...ber:/home/acme/git/linux#

By now I think there are other tools that use that python binding out in
the wild and it is packaged by distros as:

root@...ber:/home/acme/git/linux# dnf search python3-perf
============================================================================= Name Exactly Matched: python3-perf ==============================================================================
python3-perf.x86_64 : Python bindings for apps which will manipulate perf events
root@...ber:/home/acme/git/linux# 

Having these bindings opens the doors for projects to use these
libraries provided via the kernel sources, allowing other languages to
be used than C.

Now we have multiple libraries in the kernel sources, if we could have a
convenient way to provide those bindings for python, Go, etc, that could
help more projects while reducing the cost of supporting those bindings
by having more people capable of fixing up when needed as we would be
using a common way of providing such bindings.

Cheers,

- Arnaldo

Powered by blists - more mailing lists