linux-kernel - Re: [PATCH 3/3] objtool: move tools/objtool/ to scripts/objtool/

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Tue, 5 Mar 2019 08:46:25 -0600
From:   Josh Poimboeuf <jpoimboe@...hat.com>
To:     Masahiro Yamada <yamada.masahiro@...ionext.com>
Cc:     Ingo Molnar <mingo@...hat.com>,
        Peter Zijlstra <peterz@...radead.org>,
        Thomas Gleixner <tglx@...utronix.de>,
        Douglas Anderson <dianders@...omium.org>,
        Robin Meijboom <robin@...jboom.info>,
        Borislav Petkov <bp@...en8.de>,
        "H . Peter Anvin" <hpa@...or.com>, x86@...nel.org,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        Sam Ravnborg <sam@...nborg.org>, linux-kbuild@...r.kernel.org,
        linux-kernel@...r.kernel.org,
        Michal Marek <michal.lkml@...kovi.net>
Subject: Re: [PATCH 3/3] objtool: move tools/objtool/ to scripts/objtool/

On Tue, Mar 05, 2019 at 02:48:16PM +0900, Masahiro Yamada wrote:
> Tools that are globally needed for building the kernel are supposed to
> be put in the scripts/ directory, but objtool is the exceptional one.
> 
> People were confused by objtool's Makefile since most of tools run on
> the target whereas objtool runs on the host machine. Some time ago,
> Douglas Anderson suggested to move tools/objtool/ to scripts/tools/
> (https://patchwork.kernel.org/patch/9983535/#20996057), but we did
> not go as far as submitting a patch at that time.
> 
> Recently, Robin Meijboom reported that the generated files for objtool
> were never cleaned up. (https://patchwork.kernel.org/patch/10813797/)
> 
> So, I looked into this to fix the distortion. Move tools/objtool/ to
> scripts/objtool/, and rewrite Makefiles in the Kbuild-ish manner.
> 
> I also cleaned up the top-level Makefile by moving the libelf check
> scripts/objtool/Makefile.
>
> I see some benefits in this.
> 
> [1] generated files under scripts/ are cleaned up by 'make mrproper'
> 
> Currently, the generated files under tools/objtool/ are not cleaned up
> since the build system under tools/ is a complete different world.
> 
> If we want to clean them up, it should be done by 'make mrproper'
> instead of 'make clean' since objtool is needed for building external
> modules.
> 
> This commit naturally solves the issue since Kbuild cleans under
> scripts/ by 'make mrproper'.

Yes, but this can be solved instead with a one-line patch.

> [2] build log looks nicer
> 
> Currently, the build log under tools/objtool/ shows absolute paths, and
> displays 'DESCEND  objtool' every time for incremental building.
> 
> Switching over to the standard Kbuild scheme will improve the log.
> 
>   Before:
> 
>   DESCEND  objtool
>   HOSTCC   /home/masahiro/workspace/linux/tools/objtool/fixdep.o
>   HOSTLD   /home/masahiro/workspace/linux/tools/objtool/fixdep-in.o
>   LINK     /home/masahiro/workspace/linux/tools/objtool/fixdep
>   CC       /home/masahiro/workspace/linux/tools/objtool/exec-cmd.o
>   CC       /home/masahiro/workspace/linux/tools/objtool/help.o
>   CC       /home/masahiro/workspace/linux/tools/objtool/pager.o
>   CC       /home/masahiro/workspace/linux/tools/objtool/parse-options.o
>   CC       /home/masahiro/workspace/linux/tools/objtool/run-command.o
>   CC       /home/masahiro/workspace/linux/tools/objtool/sigchain.o
>   CC       /home/masahiro/workspace/linux/tools/objtool/subcmd-config.o
>   LD       /home/masahiro/workspace/linux/tools/objtool/libsubcmd-in.o
>   AR       /home/masahiro/workspace/linux/tools/objtool/libsubcmd.a
>   GEN      /home/masahiro/workspace/linux/tools/objtool/arch/x86/lib/inat-tables.c
>   CC       /home/masahiro/workspace/linux/tools/objtool/arch/x86/decode.o
>   LD       /home/masahiro/workspace/linux/tools/objtool/arch/x86/objtool-in.o
>   CC       /home/masahiro/workspace/linux/tools/objtool/builtin-check.o
>   CC       /home/masahiro/workspace/linux/tools/objtool/builtin-orc.o
>   CC       /home/masahiro/workspace/linux/tools/objtool/check.o
>   CC       /home/masahiro/workspace/linux/tools/objtool/orc_gen.o
>   CC       /home/masahiro/workspace/linux/tools/objtool/orc_dump.o
>   CC       /home/masahiro/workspace/linux/tools/objtool/elf.o
>   CC       /home/masahiro/workspace/linux/tools/objtool/special.o
>   CC       /home/masahiro/workspace/linux/tools/objtool/objtool.o
>   CC       /home/masahiro/workspace/linux/tools/objtool/libstring.o
>   CC       /home/masahiro/workspace/linux/tools/objtool/str_error_r.o
>   LD       /home/masahiro/workspace/linux/tools/objtool/objtool-in.o
>   LINK     /home/masahiro/workspace/linux/tools/objtool/objtool
> 
>   After:
> 
>   HOSTCC  scripts/objtool/builtin-check.o
>   HOSTCC  scripts/objtool/builtin-orc.o
>   HOSTCC  scripts/objtool/check.o
>   HOSTCC  scripts/objtool/orc_gen.o
>   HOSTCC  scripts/objtool/orc_dump.o
>   HOSTCC  scripts/objtool/elf.o
>   HOSTCC  scripts/objtool/special.o
>   HOSTCC  scripts/objtool/objtool.o
>   HOSTCC  scripts/objtool/libstring.o
>   HOSTCC  scripts/objtool/str_error_r.o
>   HOSTCC  scripts/objtool/exec-cmd.o
>   HOSTCC  scripts/objtool/pager.o
>   HOSTCC  scripts/objtool/parse-options.o
>   HOSTCC  scripts/objtool/run-command.o
>   HOSTCC  scripts/objtool/sigchain.o
>   HOSTCC  scripts/objtool/subcmd-config.o
>   AWK     scripts/objtool/arch/x86/lib/inat-tables.c
>   HOSTCC  scripts/objtool/arch/x86/decode.o
>   HOSTLD  scripts/objtool/objtool

That does look better.

> [3] simplify distro package script
> 
> The special handling in scripts/package/buildeb is no longer needed
> since all host programs are copied to linux-headers packages.

I do see the benefits.  In many ways objtool is a more natural fit in
the scripts dir.  But there are also some downsides to this change:

- This will make it harder to package objtool as a standalone tool.  It
  has a lot of functionality that could be useful to other non-kernel
  projects.

- It shares libsubcmd with perf.  Including the subcmd .c files from
  tools is hacky.

- It's disruptive: it will break all the out-of-tree distro kernel
  packaging scripts which now look for objtool in tools.

You're right that objtool isn't a natural fit in tools, because it's
also used as part of the build.  But it's not a natural fit in scripts
either.  I think we've resolved most of those issues and it seems to be
working well these days.

So instead of disrupting everything because "make mrproper" doesn't
work, I think I'd rather just do the following:

diff --git a/Makefile b/Makefile
index ac5ac28a24e9..7e6696c9b862 100644
--- a/Makefile
+++ b/Makefile
@@ -1364,7 +1364,7 @@ PHONY += $(mrproper-dirs) mrproper
 $(mrproper-dirs):
 	$(Q)$(MAKE) $(clean)=$(patsubst _mrproper_%,%,$@)
 
-mrproper: clean $(mrproper-dirs)
+mrproper: clean $(mrproper-dirs) tools/objtool_clean
 	$(call cmd,rmdirs)
 	$(call cmd,rmfiles)