Date: Fri, 28 Jun 2024 14:54:40 -0700
From: Brian Norris <briannorris@...omium.org>
To: Masahiro Yamada <masahiroy@...nel.org>
Cc: linux-kbuild@...r.kernel.org, Peter Zijlstra <peterz@...radead.org>,
	Josh Poimboeuf <jpoimboe@...hat.com>, linux-kernel@...r.kernel.org,
	Sami Tolvanen <samitolvanen@...gle.com>,
	Nathan Chancellor <nathan@...nel.org>,
	Nicolas Schier <nicolas@...sle.eu>
Subject: Re: [PATCH] Makefile: add comment to discourage tools/* addition for
 kernel builds

Hi Masahiro,

On Thu, Jun 27, 2024 at 04:02:46AM +0900, Masahiro Yamada wrote:
> > (Side note: I hope I haven't placed undue burden on you; I understood
> > you don't maintain tools/ and that it didn't use Kbuild. I only "poked"
> > you because the original bug report I was replying to had you and
> > linux-kbuild on CC already. And I appreciate your engagement, even if
> > the bugs are due to intentional forking.)
> 
> I did not mean to express my complaint particularly with the previous thread.

Great!

> It is not the first time that the tools/ build issue arose.
> 
> 
> I will drop the references to the threads.

That's not necessary, IMO. I'm not offended at all by any link to my
post. Links are useful, to add color to the problems involved. I was
just hoping to clarify that I never hoped Kbuild folks to solve
non-Kbuild problems.

> See commit ea01fa9f63aef, which did not get Ack from any Kbuild
> maintainer, and caused subsequent troubles, and the benefit
> of which I still do not understand.

Benefit: if I'm reading right, you've explained it yourself below?
Without this commit, it'd be harder to integrate the non-Kbuild objtool
into the Kbuild system.

But I see that it has a lot more downsides and rough edges too.

> Supporting "make tools/perf" in addition to "make -C tools perf"
> only saved a few characters to type.
> 
> 
> So, the problem remains, unless I revert ea01fa9f63aef.
> 
> I decided to not care about it too much, as long as
> such tools are not used during the kernel build.
> 
> I am really worried about objtool and resolve_btfids,
> as these two are used for building the kernel.

And of course, we ($JOB) have 2 build-flake bugs open, one for each of
those...

> >   for all sorts of kept-in-the-kernel-tree host tools? Is the
> >   recommendation to "use Kbuild" or to "avoid putting your tool in
> >   tools/*"? Is it possible (recommended?) to plumb Kbuild stuff into
> >   tools/, even if other parts won't migrate?
> 
> 
> I do not know.
> 
> They are different build systems with different designs.
> 
> Kbuild always works in the top of the output directory.
> Kbuild changes the working directory at most once if O= is given,
> but otherwise, it never changes the working directory during the build.
> 
> 
> The tools/ build system changes the working directory every time
> it invokes a new Make, and compiles the tool in its source directory.
> 
> 
> I do not know if all tools want to Kbuild.
> (the same applied to kselftest)

Yeah, from what I've tried to read off old threads, there are competing
design goals. objtool folks claimed they want to be as self-contained as
possible, with a local 'make' target, and relatively easy ability to
pull their tree out for independent usage outside the kernel.

> I think I can convert objtool and resolve_btfids to the Kbuild way.

That would make sense to me. I suspect that vastly more people build the
kernel, compared to those that want to use these tools/
independently of the kernel build. And the rough edges between them
cause real trouble.

> > As is, I can't tell if this is telling people to avoid adding new stuff
> > to tools/ entirely, or just to only add to tools/ if you're able to
> > remain completely isolated from the rest of the kernel build -- as soon
> > as you want to play some part in the Kbuild-covered part of the tree,
> > you need to use Kbuild.
> 
> 
> See the code in the top Makefile.
> 
> 'prepare' depends on tools/objtool and tools/bpf/resolve_btfids.
> 
> If other tools are not prerequisites of 'scripts',
> Kbuild will not compile them.

Sure. So I surmise you're choosing more of my latter -- that it's OK to
use tools/ if you don't want to integrate in the top-level build.

Sounds good to me.

> Nicolas suggested a link to Documentation/kbuild/makefiles.rst
> 
> We can combine the two.
> 
> 
> # The tools build system is not a part of Kbuild and tends to introduce
> # its own unique issues. If you need to integrate a new tool into Kbuild,
> # please consider locating that tool outside the tools/ tree and using the
> # standard Kbuild "hostprogs" syntax instead of adding a new tools/* entry
> # here. See Documentation/kbuild/makefiles.rst for details.

Looks good, thanks! Repeated:

Reviewed-by: Brian Norris <briannorris@...omium.org>

Powered by blists - more mailing lists