linux-kernel - Re: [PATCH] kbuild: check arch/$(SRCARCH)/include/generated before out-of-tree build

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CAMuHMdU7OY=FwWke6LqKtsMssN1ACJe15ODOp+tdkcZWoFaovw@mail.gmail.com>
Date:   Tue, 9 Jul 2019 15:05:25 +0200
From:   Geert Uytterhoeven <geert@...ux-m68k.org>
To:     Masahiro Yamada <yamada.masahiro@...ionext.com>
Cc:     linux-kbuild <linux-kbuild@...r.kernel.org>,
        Michal Marek <michal.lkml@...kovi.net>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH] kbuild: check arch/$(SRCARCH)/include/generated before
 out-of-tree build

Hi Yamada-san,

On Tue, Jul 9, 2019 at 2:59 PM Masahiro Yamada
<yamada.masahiro@...ionext.com> wrote:
> On Tue, Jul 9, 2019 at 9:45 PM Geert Uytterhoeven <geert@...ux-m68k.org> wrote:
> > On Tue, Jul 9, 2019 at 2:22 PM Masahiro Yamada
> > <yamada.masahiro@...ionext.com> wrote:
> > > On Tue, Jul 9, 2019 at 5:31 PM Geert Uytterhoeven <geert@...ux-m68k.org> wrote:
> > > > On Sun, Apr 7, 2019 at 12:45 PM Masahiro Yamada
> > > > <yamada.masahiro@...ionext.com> wrote:
> > > > > After cross-compiling the kernel, "make mrproper" should be executed
> > > > > with the proper ARCH= option. Otherwise, stale objects will remain
> > > > > under arch/$(SRCARCH)/.
> > > > >
> > > > > One bad scenario is like this:
> > > > >
> > > > >   $ make ARCH=arm defconfig all   # cross-compile the kernel for arm
> > > > >   $ make mrproper                 # mrproper for host-arch (i.e. x86)
> > > > >   $ make ARCH=arm O=build_dir defconfig all
> > > > >
> > > > > If you miss ARCH= for mrproper and cross-compile the kernel with O=
> > > > > and ARCH= options, Kbuild will happily start to build, but may fail
> > > > > due to stale objects in the srctree.
> > > > >
> > > > > If $(srctree)/arch/$(SRCARCH)/include/generated/ exists, let's stop
> > > > > the out-of-tree build. To detect this, mrproper should clean only
> > > > > arch/$(SRCARCH)/include/generated/.
> > > > >
> > > > > Signed-off-by: Masahiro Yamada <yamada.masahiro@...ionext.com>
> > > > > ---
> > > > >
> > > > >  Makefile | 6 ++++--
> > > > >  1 file changed, 4 insertions(+), 2 deletions(-)
> > > > >
> > > > > diff --git a/Makefile b/Makefile
> > > > > index 10643c3..17945ce 100644
> > > > > --- a/Makefile
> > > > > +++ b/Makefile
> > > > > @@ -1091,7 +1091,9 @@ PHONY += prepare archprepare prepare1 prepare3
> > > > >  prepare3: include/config/kernel.release
> > > > >  ifneq ($(srctree),.)
> > > > >         @$(kecho) '  Using $(srctree) as source for kernel'
> > > > > -       $(Q)if [ -f $(srctree)/.config -o -d $(srctree)/include/config ]; then \
> > > > > +       $(Q)if [ -f $(srctree)/.config -o \
> > > > > +                -d $(srctree)/include/config -o \
> > > > > +                -d $(srctree)/arch/$(SRCARCH)/include/generated ]; then \
> > > > >                 echo >&2 "  $(srctree) is not clean, please run 'make mrproper'"; \
> > > > >                 echo >&2 "  in the '$(srctree)' directory.";\
> > > >
> > > > This took me a bit to find out what was wrong...
> > > >
> > > > Usually I don't run "make mrproper", as it removes files I may want to
> > > > keep (e.g. tags).  Hence I ran "git ls-files -o | grep m68k | xargs rm"
> > > > (I usually build in separate output directories), confirmed with "git
> > > > ls-files -o" there were no remaining build artefacts, and was surprised
> > > > to discover I still got the error message above?!?
> > > >
> > > > Apparently arch/m68k/include/generated was still present, but as "git
> > > > ls-files -o" only shows files, not directories, it was not listed.
> > > > Perhaps the directory checks above can be changed to directory exists
> > > > _and_ is not empty?
> > >
> > > No.
> > >
> > > Since you did not run mrproper,
> > > Kbuild _correctly_ showed error.
> > > This is the expected and correct behavior. :)
> > >
> > > The upstream kernel is not a place
> > > to be customized for your workflow. Sorry.
> > >
> > >
> > > Every developer has a set of handy custom commands.
> > >
> > > Since you are already running a long command,
> > > why don't you add one more line, and put in ~/.bash_aliases or somewhere?
> > >
> > > my_mrproper()
> > > {
> > >     git ls-files -o | grep m68k | xargs rm
> > >     rm -rf arch/m68k/include/generated
> > > }
> >
> > Please note this was not part of my standard workflow, so I don't have a
> > script for it.  I just happened to had done a quick test build in my
> > kernel source tree repository before, and had forgotten about that.
> >
> > So IMHO this is more of a usability issue: it is difficult to find out
> > what is wrong, and how to solve it, as "git ls-files -o" doesn't give a
> > clue.  And running "make mrproper" doesn't help.
> >
> > Perhaps the message should be changed to
> >
> >     $(srctree) is not clean, please run 'make ARCH=$(SRCARCH) mrproper'"
>
> Fair enough, but 'make ARCH=$(ARCH) mrproper' please.

OK.

> Will you send a patch?

Sure, thanks!

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@...ux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds