linux-kernel - Re: [PATCH] kbuild: check arch/$(SRCARCH)/include/generated before out-of-tree build

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAK7LNATxCkEZKWknO40k-sArbwxzOjdyQdehSmdCNYjLBb7MQA@mail.gmail.com>
Date:   Tue, 9 Jul 2019 21:58:37 +0900
From:   Masahiro Yamada <yamada.masahiro@...ionext.com>
To:     Geert Uytterhoeven <geert@...ux-m68k.org>
Cc:     linux-kbuild <linux-kbuild@...r.kernel.org>,
        Michal Marek <michal.lkml@...kovi.net>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH] kbuild: check arch/$(SRCARCH)/include/generated before
 out-of-tree build

Hi Geert,

On Tue, Jul 9, 2019 at 9:45 PM Geert Uytterhoeven <geert@...ux-m68k.org> wrote:
>
> Hi Yamada-san,
>
> On Tue, Jul 9, 2019 at 2:22 PM Masahiro Yamada
> <yamada.masahiro@...ionext.com> wrote:
> > On Tue, Jul 9, 2019 at 5:31 PM Geert Uytterhoeven <geert@...ux-m68k.org> wrote:
> > > On Sun, Apr 7, 2019 at 12:45 PM Masahiro Yamada
> > > <yamada.masahiro@...ionext.com> wrote:
> > > > After cross-compiling the kernel, "make mrproper" should be executed
> > > > with the proper ARCH= option. Otherwise, stale objects will remain
> > > > under arch/$(SRCARCH)/.
> > > >
> > > > One bad scenario is like this:
> > > >
> > > >   $ make ARCH=arm defconfig all   # cross-compile the kernel for arm
> > > >   $ make mrproper                 # mrproper for host-arch (i.e. x86)
> > > >   $ make ARCH=arm O=build_dir defconfig all
> > > >
> > > > If you miss ARCH= for mrproper and cross-compile the kernel with O=
> > > > and ARCH= options, Kbuild will happily start to build, but may fail
> > > > due to stale objects in the srctree.
> > > >
> > > > If $(srctree)/arch/$(SRCARCH)/include/generated/ exists, let's stop
> > > > the out-of-tree build. To detect this, mrproper should clean only
> > > > arch/$(SRCARCH)/include/generated/.
> > > >
> > > > Signed-off-by: Masahiro Yamada <yamada.masahiro@...ionext.com>
> > > > ---
> > > >
> > > >  Makefile | 6 ++++--
> > > >  1 file changed, 4 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/Makefile b/Makefile
> > > > index 10643c3..17945ce 100644
> > > > --- a/Makefile
> > > > +++ b/Makefile
> > > > @@ -1091,7 +1091,9 @@ PHONY += prepare archprepare prepare1 prepare3
> > > >  prepare3: include/config/kernel.release
> > > >  ifneq ($(srctree),.)
> > > >         @$(kecho) '  Using $(srctree) as source for kernel'
> > > > -       $(Q)if [ -f $(srctree)/.config -o -d $(srctree)/include/config ]; then \
> > > > +       $(Q)if [ -f $(srctree)/.config -o \
> > > > +                -d $(srctree)/include/config -o \
> > > > +                -d $(srctree)/arch/$(SRCARCH)/include/generated ]; then \
> > > >                 echo >&2 "  $(srctree) is not clean, please run 'make mrproper'"; \
> > > >                 echo >&2 "  in the '$(srctree)' directory.";\
> > >
> > > This took me a bit to find out what was wrong...
> > >
> > > Usually I don't run "make mrproper", as it removes files I may want to
> > > keep (e.g. tags).  Hence I ran "git ls-files -o | grep m68k | xargs rm"
> > > (I usually build in separate output directories), confirmed with "git
> > > ls-files -o" there were no remaining build artefacts, and was surprised
> > > to discover I still got the error message above?!?
> > >
> > > Apparently arch/m68k/include/generated was still present, but as "git
> > > ls-files -o" only shows files, not directories, it was not listed.
> > > Perhaps the directory checks above can be changed to directory exists
> > > _and_ is not empty?
> >
> > No.
> >
> > Since you did not run mrproper,
> > Kbuild _correctly_ showed error.
> > This is the expected and correct behavior. :)
> >
> > The upstream kernel is not a place
> > to be customized for your workflow. Sorry.
> >
> >
> > Every developer has a set of handy custom commands.
> >
> > Since you are already running a long command,
> > why don't you add one more line, and put in ~/.bash_aliases or somewhere?
> >
> > my_mrproper()
> > {
> >     git ls-files -o | grep m68k | xargs rm
> >     rm -rf arch/m68k/include/generated
> > }
>
> Please note this was not part of my standard workflow, so I don't have a
> script for it.  I just happened to had done a quick test build in my
> kernel source tree repository before, and had forgotten about that.
>
> So IMHO this is more of a usability issue: it is difficult to find out
> what is wrong, and how to solve it, as "git ls-files -o" doesn't give a
> clue.  And running "make mrproper" doesn't help.
>
> Perhaps the message should be changed to
>
>     $(srctree) is not clean, please run 'make ARCH=$(SRCARCH) mrproper'"

Fair enough, but 'make ARCH=$(ARCH) mrproper' please.

Will you send a patch?




-- 
Best Regards
Masahiro Yamada