linux-kernel - Re: [RFC PATCH 0/3] Introduce a C extension module to allow libperf usage from python

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <owus55cwihmlvqjcphymjvfinpag3pem4qnn5xkkk3wl6o3ffs@tgycw7azoz2p>
Date: Fri, 14 Mar 2025 13:50:43 +0530
From: Gautam Menghani <gautam@...ux.ibm.com>
To: Ian Rogers <irogers@...gle.com>
Cc: "John B. Wyatt IV" <jwyatt@...hat.com>, peterz@...radead.org,
        mingo@...hat.com, acme@...nel.org, namhyung@...nel.org,
        mark.rutland@....com, alexander.shishkin@...ux.intel.com,
        jolsa@...nel.org, adrian.hunter@...el.com, kan.liang@...ux.intel.com,
        linux-perf-users@...r.kernel.org, linux-kernel@...r.kernel.org,
        maddy@...ux.ibm.com
Subject: Re: [RFC PATCH 0/3] Introduce a C extension module to allow libperf
 usage from python

On Thu, Mar 13, 2025 at 09:01:10AM -0700, Ian Rogers wrote:
> On Thu, Mar 13, 2025 at 7:12 AM John B. Wyatt IV <jwyatt@...hat.com> wrote:
> >
> > Hello Gautam
> >
> > On Thu, Mar 13, 2025 at 01:21:21PM +0530, Gautam Menghani wrote:
> > > In this RFC series, we are introducing a C extension module to allow
> > > python programs to call the libperf API functions. Currently libperf can
> > > be used by C programs, but expanding the support to python is beneficial
> > > for python users.
> > >
> > > The structure of the patch series is as follows:
> > > 1. Patch 1 : Create wrappers for the perf structs which are used by
> > > examples/counting.c
> > >
> > > 2. Patch 2: Create the C extension module that maps and exposes the
> > > libperf functions to python programs
> >
> > May I ask why you are not using SWIG? With libcpupower the kernel has
> > already been using SWIG to generate Python bindings for a C user-space API.
> >
> > This has several advantages including a much smaller footprint (you only
> > need to copy the header definitions into a .swg file), can generate for
> > several languages (Perl, Ruby, Java and C#), and SWIG is an active,
> > tested, and mature piece of software code that has been around for
> > almost as long as the Linux kernel.
> >
> > Python bindings including the makefile as an example:
> > https://elixir.bootlin.com/linux/v6.13.6/source/tools/power/cpupower/bindings/python
> >
> > How to use the bindings in a script:
> > https://elixir.bootlin.com/linux/v6.13.6/source/tools/power/cpupower/bindings/python/test_raw_pylibcpupower.py
> >
> > Original discussion:
> > https://lore.kernel.org/linux-pm/20240724221122.54601-1-jwyatt@redhat.com/
> >
> > SWIG has been pretty useful as it helped me find two issues in the
> > libcpupower API that have been around for over 10 years:
> > https://lore.kernel.org/linux-pm/20240905021916.15938-1-jwyatt@redhat.com/T/#mf04b4ba93f79fe68c20c1d88d8ed966164a1c7d7
> > https://lore.kernel.org/linux-pm/20250305210901.24177-1-jwyatt@redhat.com/
> 
> So I think we should probably get rid of libperf and re-integrate it
> back into the perf code. There are issues in the code, like removing a
> list element:
> https://web.git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/lib/perf/evlist.c?h=tmp.perf-tools-next#n58
> just leaks the removed element. Deleting the element means that the
> container evsel, rather than the libperf perf_evsel, leaks things like
> the name variable. We can add yet more call backs and complexity but
> I'm not sure what we're winning. Perhaps we can move things the other
> way, perf code into libperf, like machine, session, pmus, .. I'd
> prefer if we were to do that we refactored the code and adopted the
> same license as libbpf as both libraries have similar packaging
> issues. The viral GPLv2 on libperf is something of an issue. Perhaps
> we can also migrate this code to Rust.
> 
> SWIG is fine, there is also CLIF, I'm way of dependencies as even a
> python dependency in the perf build is optional.
> 
> We already have perf python bindings, and I've been working to extend
> those for example in:
> https://lore.kernel.org/lkml/20250228222308.626803-1-irogers@google.com/
> and I've been working to expand those for things like hybrid CPUs. It
> seems a shame to reinvent all of that logic again on top of libperf.

Thanks for the feedback and pointers. We'll look more into this
approach.

Thanks,
Gautam