linux-kernel - Re: [PATCH] Makefile: add comment to discourage tools/* addition for kernel builds

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <ZnSkmmpCY2Aj5VpU@google.com>
Date: Thu, 20 Jun 2024 14:52:26 -0700
From: Brian Norris <briannorris@...omium.org>
To: Masahiro Yamada <masahiroy@...nel.org>
Cc: linux-kbuild@...r.kernel.org, Peter Zijlstra <peterz@...radead.org>,
	Josh Poimboeuf <jpoimboe@...hat.com>, linux-kernel@...r.kernel.org,
	Sami Tolvanen <samitolvanen@...gle.com>,
	Nathan Chancellor <nathan@...nel.org>,
	Nicolas Schier <nicolas@...sle.eu>
Subject: Re: [PATCH] Makefile: add comment to discourage tools/* addition for
 kernel builds

On Wed, Jun 19, 2024 at 03:21:42PM +0900, Masahiro Yamada wrote:
> Kbuild provides scripts/Makefile.host to build host programs used for
> building the kernel. Unfortunately, there are two exceptions that opt
> out of Kbuild. The build system under tools/ is a cheesy replica, and
> is always a disaster. I was recently poked about a problem in the tools
> build issue, which I do not maintain (and nobody maintains). [1]

(Side note: I hope I haven't placed undue burden on you; I understood
you don't maintain tools/ and that it didn't use Kbuild. I only "poked"
you because the original bug report I was replying to had you and
linux-kbuild on CC already. And I appreciate your engagement, even if
the bugs are due to intentional forking.)

But anyway, I agree that clearer documentation and recommendations could
be helpful here. To that end, some dumb questions below, as I'm not sure
if this fully serves its purpose as-is:

> Without a comment, somebody might believe this is the right location
> because that is where objtool lives, even when a more robust Kbuild
> syntax satisfies their needs. [2]
> 
> [1]: https://lore.kernel.org/linux-kbuild/ZnIYWBgrJ-IJtqK8@google.com/T/#m8ece130dd0e23c6f2395ed89070161948dee8457
> [2]: https://lore.kernel.org/all/20240618200501.GA1611012@google.com/
> 
> Signed-off-by: Masahiro Yamada <masahiroy@...nel.org>
> ---
> 
>  Makefile | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/Makefile b/Makefile
> index 471f2df86422..ba070596ad4e 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1331,6 +1331,11 @@ prepare: tools/bpf/resolve_btfids
>  endif
>  endif
>  
> +# README
> +# The tools build system is not a part of Kbuild. Before adding yet another
> +# tools/* here, please consider if the standard "hostprogs" syntax satisfies
> +# your needs.
> +

Some clarifying questions / statements-as-questions:

* nothing in tools/ uses Kbuild, right? (even stuff that uses KBUILD_*
  names is just an imitative port, right?)
* not everything in tools/ is actually promoted to a high-level target,
  that affects this top-level Makefile. Are you only concerned about
  stuff that pretends to be integrated in the top-level kernel Makefile?
  (If not, then it seems like placing the README comments only in this
  Makefile is a poor choice.)
* is the "standard hostprogs" recommendation a general recommendation,
  for all sorts of kept-in-the-kernel-tree host tools? Is the
  recommendation to "use Kbuild" or to "avoid putting your tool in
  tools/*"? Is it possible (recommended?) to plumb Kbuild stuff into
  tools/, even if other parts won't migrate?

As is, I can't tell if this is telling people to avoid adding new stuff
to tools/ entirely, or just to only add to tools/ if you're able to
remain completely isolated from the rest of the kernel build -- as soon
as you want to play some part in the Kbuild-covered part of the tree,
you need to use Kbuild.

If I'm inferring the right answers to the above, then maybe an improved
wording could be something like:

"The tools build system is not a part of Kbuild and tends to introduce
its own unique issues. If you need to integrate a new tool into Kbuild,
please consider locating that tool outside the tools/ tree and using the
standard Kbuild "hostprogs" syntax instead of adding a new tools/* entry
here."

It's possible I'm playing mental acrobatics here in my reading too.

Either way, I think this is a good trajectory:

Reviewed-by: Brian Norris <briannorris@...omium.org>

Regards,
Brian

>  PHONY += resolve_btfids_clean
>  
>  resolve_btfids_O = $(abspath $(objtree))/tools/bpf/resolve_btfids
> -- 
> 2.43.0
>