linux-kernel - [RFC PATCH] kbuild: Add option to generate a Compilation Database

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:   Thu,  6 Jun 2019 14:30:03 -0600
From:   Raul E Rangel <rrangel@...omium.org>
To:     yamada.masahiro@...ionext.com
Cc:     mka@...omium.org, ndesaulniers@...gle.com, zwisler@...omium.org,
        Raul E Rangel <rrangel@...omium.org>,
        Joe Lawrence <joe.lawrence@...hat.com>,
        Kees Cook <keescook@...omium.org>,
        linux-kbuild@...r.kernel.org, Petr Mladek <pmladek@...e.com>,
        linux-kernel@...r.kernel.org,
        Michal Marek <michal.lkml@...kovi.net>,
        Andy Shevchenko <andriy.shevchenko@...ux.intel.com>,
        Changbin Du <changbin.du@...el.com>,
        Tetsuo Handa <penguin-kernel@...ove.sakura.ne.jp>,
        Sri Krishna chowdary <schowdary@...dia.com>,
        Matthew Wilcox <willy@...radead.org>,
        Mikulas Patocka <mpatocka@...hat.com>,
        Andrew Morton <akpm@...ux-foundation.org>
Subject: [RFC PATCH] kbuild: Add option to generate a Compilation Database

Clang tooling requires a compilation database to figure out the build
options for each file. This enables tools like clang-tidy and
clang-check.

See https://clang.llvm.org/docs/HowToSetupToolingForLLVM.html for more
information.

Normally cmake is used to generate the compilation database, but the
linux kernel uses make. Another option is using
[BEAR](https://github.com/rizsotto/Bear) which instruments
exec to find clang invocations and generate the database that way.

Clang 4.0.0 added the -MJ option to generate the json for each
compilation unit. https://reviews.llvm.org/D27140

This patch takes advantage of the -MJ option. So it only works for
Clang.

Signed-off-by: Raul E Rangel <rrangel@...omium.org>
---
I have a couple TODOs in the code that I would like some feedback on.
Specifically why extra-y doesn't seem to work in the root Makefile.
Also, is there a way to add the correct list of prerequisites to the
compile_commands.json target?

Thanks,
Raul


 Makefile               | 20 ++++++++++++++++++++
 lib/Kconfig.debug      |  7 +++++++
 scripts/Makefile.build |  9 ++++++++-
 3 files changed, 35 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index a61a95b6b38f7..06067ee18ff64 100644
--- a/Makefile
+++ b/Makefile
@@ -1663,6 +1663,26 @@ quiet_cmd_tags = GEN     $@
 tags TAGS cscope gtags: FORCE
 	$(call cmd,tags)
 
+# Compilation Database
+# ---------------------------------------------------------------------------
+# Generates a compilation database that can be used with the LLVM tools
+ifdef CONFIG_COMPILATION_DATABASE
+
+quiet_cmd_compilation_db = GEN   $@
+cmd_compilation_db = (echo '['; \
+	find "$(@D)" -mindepth 2 -iname '*.json' -print0 | xargs -0 cat; \
+	echo ']') > "$(@D)/$(@F)"
+
+# Make sure the database is built when calling `make` without a target.
+# TODO: Using extra-y doesn't seem to work.
+_all: $(obj)/compile_commands.json
+
+# TODO: Is there a variable that contains all the object files created by
+# cmd_cc_o_c? Depending on `all` is kind of a hack
+$(obj)/compile_commands.json: all FORCE
+	$(call if_changed,compilation_db)
+endif
+
 # Scripts to check various things for consistency
 # ---------------------------------------------------------------------------
 
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index eae43952902eb..46fceb1fff3d9 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -238,6 +238,13 @@ config GDB_SCRIPTS
 	  instance. See Documentation/dev-tools/gdb-kernel-debugging.rst
 	  for further details.
 
+config COMPILATION_DATABASE
+	bool "Generate a compilation database"
+	depends on CLANG_VERSION >= 40000
+	help
+	  This creates a JSON Compilation Database (compile_commands.json)
+	  that is used by the clang tooling (clang-tidy, clang-check, etc).
+
 config ENABLE_MUST_CHECK
 	bool "Enable __must_check logic"
 	default y
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index ae9cf740633e1..0017bf397292d 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -145,8 +145,15 @@ $(obj)/%.ll: $(src)/%.c FORCE
 # The C file is compiled and updated dependency information is generated.
 # (See cmd_cc_o_c + relevant part of rule_cc_o_c)
 
+ifdef CONFIG_COMPILATION_DATABASE
+# TODO: Should we store the json in a temp variable and only copy it to the
+# final name when the content is different? In theory we could avoid having to
+# generate the compilation db if the json did not change.
+compdb_flags = -MJ $(@D)/.$(@F).json
+endif
+
 quiet_cmd_cc_o_c = CC $(quiet_modtag)  $@
-      cmd_cc_o_c = $(CC) $(c_flags) -c -o $@ $<
+      cmd_cc_o_c = $(CC) $(c_flags) $(compdb_flags) -c -o $@ $<
 
 ifdef CONFIG_MODVERSIONS
 # When module versioning is enabled the following steps are executed:
-- 
2.22.0.rc1.311.g5d7573a151-goog