linux-kernel - Re: [GIT PULL 0/7] perf/core improvements and fixes

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20151126081236.GB20999@gmail.com>
Date:	Thu, 26 Nov 2015 09:12:37 +0100
From:	Ingo Molnar <mingo@...nel.org>
To:	Jiri Olsa <jolsa@...hat.com>
Cc:	Arnaldo Carvalho de Melo <acme@...nel.org>,
	linux-kernel@...r.kernel.org, David Ahern <dsahern@...il.com>,
	Jan Kratochvil <jkratoch@...hat.com>,
	Masami Hiramatsu <masami.hiramatsu.pt@...achi.com>,
	Milian Wolff <milian.wolff@...b.com>,
	Namhyung Kim <namhyung@...nel.org>,
	Peter Zijlstra <peterz@...radead.org>,
	Taeung Song <treeze.taeung@...il.com>,
	Wang Nan <wangnan0@...wei.com>,
	Arnaldo Carvalho de Melo <acme@...hat.com>
Subject: Re: [GIT PULL 0/7] perf/core improvements and fixes

* Ingo Molnar <mingo@...nel.org> wrote:

> > Could you please share failing build output from:
> >   $ make V=1 clean install
> 
> The problem went away after I unpulled - I'll pull it again and will try to get 
> you debug output.

Bugger, it does not reproduce after I re-pulled Arnaldo's original bits. I 
re-pulled bit for bit - yet something has changed that changed the timings.

One thing I did meanwhile: after a fairly long time I git-gc'ed my Git repo 
(auto-gc is turned off) - so maybe it's the anomalous execution length of the 
util/PERF-VERSION-GEN version generation script that triggers the race?

... and indeed: if I add a 'sleep 1' to the end of util/PERF-VERSION-GEN I 
sometimes get this failure:

  LD       plugin_jbd2-in.o
  CC       plugin_hrtimer.o
  CC       plugin_hrtimer.o
  LD       plugin_hrtimer-in.o
  CC       plugin_kmem.o
fixdep: error opening depfile: ./.plugin_hrtimer.o.d: No such file or directory
/home/mingo/tip/tools/build/Makefile.build:77: recipe for target 
'plugin_hrtimer.o' failed
make[3]: *** [plugin_hrtimer.o] Error 2
Makefile:189: recipe for target 'plugin_hrtimer-in.o' failed
make[2]: *** [plugin_hrtimer-in.o] Error 2
Makefile.perf:414: recipe for target 'libtraceevent_plugins' failed
make[1]: *** [libtraceevent_plugins] Error 2
make[1]: *** Waiting for unfinished jobs....
  CC       plugin_kvm.o
  CC       builtin-sched.o
  CC       plugin_mac80211.o
  LD       plugin_kmem-in.o
  CC       util/help.o

Can you reproduce it by adding 'sleep 1' (or 'sleep 2') to the end of 
util/PERF-VERSION-GEN? (See the patch attached further below.)

The failure is sporadic even with the 'sleep 1' hack - I get a build failure 1 out 
of 3 times maybe:

triton:~/tip/tools/perf> while : ; do echo -n "$(date) "; make clean install >/tmp/FAIL.log 2>&1 && echo pass || { echo 'FAIL!'; break; } done
Thu Nov 26 09:07:12 CET 2015 pass
Thu Nov 26 09:07:23 CET 2015 FAIL!

Interestingly it does not reproduce if I use '>FAIL.log' - I need to output the 
log into tmpfs - maybe the extra VFS synchronization if logging to the same 
filesystem where the build occurs hides the race?

With V=1 it takes a lot longer to reproduce:

triton:~/tip/tools/perf> while : ; do echo -n "$(date) "; make clean install V=1 >/tmp/FAIL.log 2>&1 && echo pass || { echo 'FAIL!'; break; } done
Thu Nov 26 09:08:08 CET 2015 pass
Thu Nov 26 09:08:20 CET 2015 pass
Thu Nov 26 09:08:32 CET 2015 pass
Thu Nov 26 09:08:43 CET 2015 pass
Thu Nov 26 09:08:55 CET 2015 pass
Thu Nov 26 09:09:06 CET 2015 pass
Thu Nov 26 09:09:17 CET 2015 pass
Thu Nov 26 09:09:29 CET 2015 pass
Thu Nov 26 09:09:41 CET 2015 FAIL!

I've attached the resulting FAIL.log.

Thanks,

	Ingo

 tools/perf/util/PERF-VERSION-GEN | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/perf/util/PERF-VERSION-GEN b/tools/perf/util/PERF-VERSION-GEN
index 39f17507578d..2b94c25eb775 100755
--- a/tools/perf/util/PERF-VERSION-GEN
+++ b/tools/perf/util/PERF-VERSION-GEN
@@ -48,3 +48,4 @@ test "$VN" = "$VC" || {
 }

+sleep 1

View attachment "FAIL.log" of type "text/plain" (282255 bytes)