linux-kernel - Re: [GIT PULL] perf tools changes for v6.2: 1st batch

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Sat, 17 Dec 2022 11:50:07 -0300
From:   Arnaldo Carvalho de Melo <acme@...nel.org>
To:     Linus Torvalds <torvalds@...ux-foundation.org>
Cc:     Ingo Molnar <mingo@...nel.org>,
        Thomas Gleixner <tglx@...utronix.de>,
        Jiri Olsa <jolsa@...nel.org>,
        Namhyung Kim <namhyung@...nel.org>,
        Clark Williams <williams@...hat.com>,
        Kate Carcia <kcarcia@...hat.com>, linux-kernel@...r.kernel.org,
        linux-perf-users@...r.kernel.org,
        Adrian Hunter <adrian.hunter@...el.com>,
        Dmitrii Dolgov <9erthalion6@...il.com>,
        Ian Rogers <irogers@...gle.com>,
        James Clark <james.clark@....com>,
        Kang Minchul <tegongkang@...il.com>,
        Kan Liang <kan.liang@...ux.intel.com>,
        Kautuk Consul <kconsul@...tanamicro.com>,
        Leo Yan <leo.yan@...aro.org>,
        Masami Hiramatsu <mhiramat@...nel.org>,
        Nikita Shubin <n.shubin@...ro.com>,
        Yang Jihong <yangjihong1@...wei.com>,
        Steven Rostedt <rostedt@...dmis.org>,
        Andrii Nakryiko <andrii@...nel.org>,
        Arnaldo Carvalho de Melo <acme@...hat.com>
Subject: Re: [GIT PULL] perf tools changes for v6.2: 1st batch

Em Fri, Dec 16, 2022 at 01:41:24PM -0600, Linus Torvalds escreveu:
> On Fri, Dec 16, 2022 at 8:36 AM Arnaldo Carvalho de Melo <acme@...nel.org> wrote:

> >         This time there is a trivial resolution of a merge conflict
> > related to commit c302378bc157f6a7 ("libbpf: Hashmap interface update to
> > allow both long and void* keys/values"), that touched tools/perf/ files.
 
> Normally I'd ask you very sternly to please not resolve merge
> conflicts for me, but since I just screwed up another merge due to

I thought I was helping you... Now I know better :-/

> being on the road and not having done a full build test, I guess I
> will just be quiet this time.
 
> So the real reason for this email is that I get an error like this on
> my laptop when trying to build the perf tools:
 
>     INSTALL libbpf_headers
>   Traceback (most recent call last):
>     File "util/setup.py", line 31, in <module>
>       from setuptools import setup, Extension
>   ImportError: No module named setuptools
>   cp: cannot stat 'python_ext_build/lib/perf*.so': No such file or directory
>   make[2]: *** [Makefile.perf:651: python/perf] Error 1
>   make[2]: *** Waiting for unfinished jobs....
>   make[1]: *** [Makefile.perf:236: sub-make] Error 2
>   make: *** [Makefile:70: all] Error 2
 
> and it's not new to this pull - I've seen it before and it's something
> stupid I've done.
 
> I think it's a missing package dependency that this laptop then hits.
> But considering my previous merge failure, I'm not super-happy about
> traveling with a laptop that then can't do this build test.

My bad, I should have acted upon that immediately, oh well, fell thru
the cracks but now I have a fix in my perf/core and perf/urgent
branches.

  git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git perf/urgent

It will be in my next pull req if you prefer not to apply it unsigned or
manually from the copy below.
 
> It would be lovely to have that package dependency - whatever it may
> be - tested for explicitly. Hint hint.

And below you have it, now back to the pool, the kids are waiting :-)

Regards,

- Arnaldo

>From 71c2c68e4fadc9019d727bdcf54490c438d59cdb Mon Sep 17 00:00:00 2001
From: Arnaldo Carvalho de Melo <acme@...hat.com>
Date: Sat, 17 Dec 2022 11:37:15 -0300
Subject: [PATCH 1/1] perf python: Don't stop building if python setuptools
 isn't installed

The python3-setuptools package is needed to build the python binding, so
that one can use things like:

  # ~acme/git/perf/tools/perf/python/twatch.py
  cpu: 6, pid: 4573, tid: 2184618 { type: exit, pid: 4573, ppid: 4172, tid: 2184618, ptid: 4172, time: 12563190090107}
  cpu: 24, pid: 4573, tid: 4573 { type: fork, pid: 4573, ppid: 4573, tid: 2190991, ptid: 4573, time: 12563415289331}
  cpu: 29, pid: 4573, tid: 2190991 { type: comm, pid: 4573, tid: 2190991, comm: StreamT~ns #401 }
  cpu: 29, pid: 4573, tid: 2190991 { type: comm, pid: 4573, tid: 2190991, comm: StreamT~ns #401 }
  ^CTraceback (most recent call last):
    File "/var/home/acme/git/perf/tools/perf/python/twatch.py", line 61, in <module>
      main()
    File "/var/home/acme/git/perf/tools/perf/python/twatch.py", line 33, in main
      evlist.poll(timeout = -1)
  KeyboardInterrupt

  #

That have 'import perf;'.

But distros don't always have that python3-setuptools (or equivalent)
installed, which was breaking the build. Just check if it is installed
and emit a warning that such binding isn't being built and continue the
build without it:

With it:

  $ rpm -q python3-setuptools
  python3-setuptools-59.6.0-3.fc36.noarch
  $ rm -rf /tmp/build/perf; mkdir -p /tmp/build/perf
  $ make O=/tmp/build/perf -C tools/perf install-bin
  make: Entering directory '/var/home/acme/git/perf/tools/perf'
  <SNIP>
  ...                               libpython: [ on  ]
  <SNIP>
    GEN     /tmp/build/perf/python/perf.cpython-310-x86_64-linux-gnu.so
  <SNIP>
  $ ls -la /tmp/build/perf/python/perf.cpython-310-x86_64-linux-gnu.so
  -rwxr-xr-x. 1 acme acme 1609112 Dec 17 11:39 /tmp/build/perf/python/perf.cpython-310-x86_64-linux-gnu.so
  $

Without it:

  $ sudo rpm -e python3-setuptools
  $ rm -rf /tmp/build/perf ; mkdir -p /tmp/build/perf
  $ make O=/tmp/build/perf -C tools/perf install-bin
  make: Entering directory '/var/home/acme/git/perf/tools/perf'
  <SNIP>
  ...                               libpython: [ on  ]
  <SNIP>
  $ ls -la /tmp/build/perf/python/perf.cpython-310-x86_64-linux-gnu.so
  ls: cannot access '/tmp/build/perf/python/perf.cpython-310-x86_64-linux-gnu.so': No such file or directory
  $

Reported-by: Linus Torvalds <torvalds@...ux-foundation.org>
Cc: Adrian Hunter <adrian.hunter@...el.com>
Cc: Ian Rogers <irogers@...gle.com>
Cc: Jiri Olsa <jolsa@...nel.org>
Cc: Namhyung Kim <namhyung@...nel.org>
Signed-off-by: Arnaldo Carvalho de Melo <acme@...hat.com>
---
 tools/perf/Makefile.config | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/tools/perf/Makefile.config b/tools/perf/Makefile.config
index 83ed969b95b4a53f..c21bd6010be1384c 100644
--- a/tools/perf/Makefile.config
+++ b/tools/perf/Makefile.config
@@ -890,8 +890,13 @@ else
       else
          LDFLAGS += $(PYTHON_EMBED_LDFLAGS)
          EXTLIBS += $(PYTHON_EMBED_LIBADD)
-         PYTHON_EXTENSION_SUFFIX := $(shell $(PYTHON) -c 'from importlib import machinery; print(machinery.EXTENSION_SUFFIXES[0])')
-         LANG_BINDINGS += $(obj-perf)python/perf$(PYTHON_EXTENSION_SUFFIX)
+         PYTHON_SETUPTOOLS_INSTALLED := $(shell $(PYTHON) -c 'import setuptools;' 2> /dev/null && echo "yes" || echo "no")
+         ifeq ($(PYTHON_SETUPTOOLS_INSTALLED), yes)
+           PYTHON_EXTENSION_SUFFIX := $(shell $(PYTHON) -c 'from importlib import machinery; print(machinery.EXTENSION_SUFFIXES[0])')
+           LANG_BINDINGS += $(obj-perf)python/perf$(PYTHON_EXTENSION_SUFFIX)
+	 else
+           msg := $(warning Missing python setuptools, the python binding won't be built, please install python3-setuptools or equivalent);
+         endif
          CFLAGS += -DHAVE_LIBPYTHON_SUPPORT
          $(call detected,CONFIG_LIBPYTHON)
       endif
-- 
2.38.1