Message-ID: <1465263518.3885.9.camel@intel.com>
Date:	Tue, 7 Jun 2016 01:38:40 +0000
From:	"Zanoni, Paulo R" <paulo.r.zanoni@...el.com>
To:	"mmarek@...e.com" <mmarek@...e.com>,
	"yamada.masahiro@...ionext.com" <yamada.masahiro@...ionext.com>
CC:	"linux-kbuild@...r.kernel.org" <linux-kbuild@...r.kernel.org>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"nicolas.pitre@...aro.org" <nicolas.pitre@...aro.org>
Subject: Regression in "kbuild: fix if_change and friends to consider
 argument order"

Hi

I recently noticed that alternating between "make" and "make targz-pkg"
rebuilds the whole Kernel. This was not happening before. As a Kernel
developer, my build/install/test environment heavily relies on the fact
that "make targz-pkg" only quickly generates the tarball if everything
is already built, so this change is heavily impacting my development
environment.

I did some bisection and concluded that the first bad commit is:

commit 9c8fa9bc08f60ac657751daba9fccf828a36cfed
Author: Masahiro Yamada
Date:   Sat May 7 15:48:26 2016 +0900
    kbuild: fix if_change and friends to consider argument order

I also verified that if I just revert this commit on top of the
most recent tree it goes back to the usual behavior.

I read the commit message and it seems that some unneeded rebuilds are
somewhat expected, but I can't understand why such a change in the
command line like the one I did triggers everything to be rebuilt.
IMHO, it really shouldn't. I also wonder that maybe the regression I'm
experiencing was not expected in the original change, so maybe there's
a way to keep the original improvement caused by the mentioned patch
without the regression I'm experiencing.

How to reproduce (exact commands I used at every bisect step):

$ make tinyconfig
$ time make -j4 V=2 # this should build things
$ time make -j4 V=2 # just to make sure nothing will be rebuilt
$ time make -j4 V=2 targz-pkg

In bad commits, the last command will trigger a rebuild of the whole
Kernel, mentioning "due to command line change" as a reason to rebuild
every object file. In good commits, the last command will finish after
a few seconds, not rebuilding anything, just giving us a nice tarball.

Also notice that if you do wait for the whole targz-pkg build, and
later go back to "make -j4 V=2" (without targz-pkg), it will, again,
rebuild everything, which is also not the old behavior.

After some investigation, it looks like the .obj.cmd files are what's
compared here, so let me provide you an example that will hopefully
help you debug:

$ time make -j4 V=2
$ cp init/.main.o.cmd /tmp/main-before.cmd
$ time make -j4 V=2 targz-pkg
$ diff -Nrup /tmp/main-before.cmd init/.main.o.cmd > /tmp/cmd-
diff.patch

You can access the diff file here:
  https://people.freedesktop.org/~pzanoni/cmd-diff.patch

If you take a look, you'll see that the only difference is that the cmd
file generated during targz-pkg contains these extra chars in the
cmd_init/main.o declaration:
  -nostdinc -isystem /usr/lib/gcc/x86_64-linux-gnu/5/include

But please notice that these arguments are repeated! The main-
before.cmd file contains those arguments only once, while the new file
contains them twice. I don't know why "make targz-pkg" makes these
arguments appear twice. Maybe fixing this would be a better way to
solve the regression instead of just reverting the "first bad commit"?

I didn't debug the problem further. I'm hoping the info I provided is
enough for you to keep things moving, but if you have any patches or
tips on where I could continue digging, please tell me. I suppose the
problem should be easily reproducible everywhere.

In the meantime, since I can't pay the price of 2 full rebuilds for
every tested Kernel, I suppose I'll be carrying a local revert, but I'd
be really happy to see an upstream solution.

Thanks,
Paulo

Powered by blists - more mailing lists