linux-kernel - Re: [PATCH] perf: build reproducibility improvements

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <YE9TAY05BtXbuHxq@krava>
Date:   Mon, 15 Mar 2021 13:28:49 +0100
From:   Jiri Olsa <jolsa@...hat.com>
To:     Denys Zagorui <dzagorui@...co.com>
Cc:     peterz@...radead.org, mingo@...hat.com, acme@...nel.org,
        mark.rutland@....com, alexander.shishkin@...ux.intel.com,
        namhyung@...nel.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH] perf: build reproducibility improvements

On Fri, Mar 12, 2021 at 03:17:00PM +0000, Denys Zagorui wrote:
> This patch helps to make perf build more reproducible
> 
> It seems there is some need to have an ability to invoke
> perf from build directory without installation
> (84cfac7f05e1: perf tools: Set and pass DOCDIR to builtin-report.c)
> DOCDIR contains an absolute path to kernel source directory.
> In such case this path can be determined in runtime by using
> /proc/self/exe link. In case of building perf with O=
> Documentation/tips.txt can be copied to output directory.
> 
> There is also python binding test where PYTHONPATH is used to store
> absolute path to python/perf.so library. This path can be
> also determined in runtime.
> 
> bison stores full paths in generated files. This can be
> remapped by using --file-prefix-map option that is available
> starting from version 3.6.3.
> 
> Signed-off-by: Denys Zagorui <dzagorui@...co.com>

hi,
I'm getting build error:

[jolsa@...va perf]$ make JOBS=1
  BUILD:   Doing 'make -j1' parallel build
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h'
diff -u tools/include/uapi/linux/kvm.h include/uapi/linux/kvm.h
Makefile.config:1026: No openjdk development package found, please install JDK package, e.g. openjdk-8-jdk, java-1.8.0-openjdk-devel
cp: '/home/jolsa/kernel/linux-perf/tools/perf/Documentation/tips.txt' and 'Documentation/tips.txt' are the same file
  BISON    util/parse-events-bison.c
bison: unrecognized option '--file-prefix-map=='
Try 'bison --help' for more information.
make[4]: *** [util/Build:211: util/parse-events-bison.c] Error 1
make[3]: *** [/home/jolsa/kernel/linux-perf/tools/build/Makefile.build:139: util] Error 2
make[2]: *** [Makefile.perf:653: perf-in.o] Error 2
make[1]: *** [Makefile.perf:236: sub-make] Error 2
make: *** [Makefile:70: all] Error 2


SNIP

> diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
> index f6e609673de2..2bebff69b064 100644
> --- a/tools/perf/Makefile.perf
> +++ b/tools/perf/Makefile.perf
> @@ -627,6 +627,9 @@ $(OUTPUT)python/perf.so: $(PYTHON_EXT_SRCS) $(PYTHON_EXT_DEPS) $(LIBTRACEEVENT_D
>  	  --quiet build_ext; \
>  	cp $(PYTHON_EXTBUILD_LIB)perf*.so $(OUTPUT)python/
>  
> +_dummy := $(shell [ -d '$(OUTPUT)Documentation' ] || mkdir -p '$(OUTPUT)Documentation' && \
> +		cp '$(srcdir)/Documentation/tips.txt' '$(OUTPUT)Documentation/')
> +
>  please_set_SHELL_PATH_to_a_more_modern_shell:
>  	$(Q)$$(:)
>  
> diff --git a/tools/perf/builtin-report.c b/tools/perf/builtin-report.c
> index 2a845d6cac09..d9441055357e 100644
> --- a/tools/perf/builtin-report.c
> +++ b/tools/perf/builtin-report.c
> @@ -610,12 +610,27 @@ static int report__browse_hists(struct report *rep)
>  	struct perf_session *session = rep->session;
>  	struct evlist *evlist = session->evlist;
>  	const char *help = perf_tip(system_path(TIPDIR));
> +	char *exec_path;
> +	char *docdir;
>  
>  	if (help == NULL) {
>  		/* fallback for people who don't install perf ;-) */
> -		help = perf_tip(DOCDIR);
> -		if (help == NULL)
> -			help = "Cannot load tips.txt file, please install perf!";
> +		exec_path = get_exec_abs_path();
> +		if (exec_path == NULL || asprintf(&docdir, "%sDocumentation", exec_path) < 0) {
> +			docdir = NULL;
> +			help = "Not enough memory or some other internal error occurred!";
> +		}

hum, do we actualy want this? I think we want the exact path
we used for compilation, no? what's the benefit?

> +
> +		if (docdir != NULL) {
> +			help = perf_tip(docdir);
> +			if (help == NULL)
> +				help = "Cannot load tips.txt file, please install perf!";
> +		}
> +
> +		if (exec_path)
> +			free(exec_path);
> +		if (docdir)
> +			free(docdir);
>  	}
>  
>  	switch (use_browser) {
> diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
> index 650aec19d490..a20098dcdbc4 100644
> --- a/tools/perf/tests/Build
> +++ b/tools/perf/tests/Build
> @@ -98,5 +98,5 @@ perf-$(CONFIG_DWARF_UNWIND) += dwarf-unwind.o
>  endif
>  
>  CFLAGS_attr.o         += -DBINDIR="BUILD_STR($(bindir_SQ))" -DPYTHON="BUILD_STR($(PYTHON_WORD))"
> -CFLAGS_python-use.o   += -DPYTHONPATH="BUILD_STR($(OUTPUT)python)" -DPYTHON="BUILD_STR($(PYTHON_WORD))"
> +CFLAGS_python-use.o   += -DPYTHON="BUILD_STR($(PYTHON_WORD))"
>  CFLAGS_dwarf-unwind.o += -fno-optimize-sibling-calls
> diff --git a/tools/perf/tests/python-use.c b/tools/perf/tests/python-use.c
> index 98c6d474aa6f..8b4865c90d5d 100644
> --- a/tools/perf/tests/python-use.c
> +++ b/tools/perf/tests/python-use.c
> @@ -8,16 +8,28 @@
>  #include <linux/compiler.h>
>  #include "tests.h"
>  #include "util/debug.h"
> +#include <subcmd/exec-cmd.h>
>  
>  int test__python_use(struct test *test __maybe_unused, int subtest __maybe_unused)
>  {
>  	char *cmd;
>  	int ret;
> +	char *exec_path = NULL;
> +	char *pythonpath;
> +
> +	exec_path = get_exec_abs_path();
> +	if (exec_path == NULL)
> +		return -1;
> +
> +	if (asprintf(&pythonpath, "%spython", exec_path) < 0)
> +		return -1;

same here, we want to be sure to use the python path
from the exact build laction no?

thanks,
jirka