Message-ID: <20250715052459.0000e119@gandalf.local.home>
Date: Tue, 15 Jul 2025 05:24:59 -0400
From: Steven Rostedt <rostedt@...dmis.org>
To: Linus Torvalds <torvalds@...ux-foundation.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@...icios.com>, Steven Rostedt
 <rostedt@...nel.org>, Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
 Christoph Hellwig <hch@...radead.org>, "Masami Hiramatsu (Google)"
 <mhiramat@...nel.org>, Peter Zijlstra <peterz@...radead.org>, Ingo Molnar
 <mingo@...nel.org>, Thomas Gleixner <tglx@...utronix.de>, linux-kernel
 <linux-kernel@...r.kernel.org>
Subject: Re: [RFC] LTTng upstreaming next steps

On Mon, 14 Jul 2025 14:04:38 -0700
Linus Torvalds <torvalds@...ux-foundation.org> wrote:

> On Mon, 14 Jul 2025 at 13:37, Steven Rostedt <rostedt@...dmis.org> wrote:
> >
> > If LTTng were to be in tree, it would be much easier to work on a strategy
> > to merge the infrastructure as the maintainers of both would have better
> > access to each other's code.  
> 
> That's not a bet I'd take.
> 
> If people haven't unified this in the last two decades, I'm noty going
> to take the argument of "hey, merge it because *then* it will be
> unified".
> 
> Because honestly, that sounds like a total fairy tale to me: "the
> princess came along and kissed the toad, and he turned into a
> beautiful price, and they lived happily ever after".
> 
> So no. I don't believe in fairy tales. Not when we have two decades of
> "that didn't happen".
> 
> If people can unify this and merge it incrementally, that's one thing.
> 
> Until then, you're just making stuff up.
> 
> "Show me the code", in other words.

We'd love to, but how is one to merge code in incremental steps when they
are not allowed to use the current infrastructure?

LTTng being out of tree means it is only allowed to use EXPORT_SYMBOL_GPL()
functions as it must be a module. Perf, ftrace and BPF are all 100% in-tree
and has no modules so they do not require any EXPORT_SYMBOL_GPL() to use
the current infrastructure.

This is the main difference between an external tracer and an external
driver. That's because other in-tree drivers and file systems all have a
module component. If ftrace, perf and BPF were modules, it would require the
infrastructure they use to all be defined as EXPORT_SYMBOL_GPL(). This
makes LTTng have a huge disadvantage as those are not modules and the
interface they use are not exported.

Christoph and Greg keep saying "this is no different than any other out of
tree code", but I'm saying due to this limitation it is. When they talk
about out of tree drivers and file systems that need to do the same, that
code has a huge advantage because it is competing with other code that are
modules and they can use the same infrastructure. Even when they are out of
tree.

Case in point, Mathieu is helping with the sframe work so that it can share
the same code that perf and ftrace will use. But as soon as we want to
allow it to use that code, it is shot down because it requires a
EXPORT_SYMBOL_GPL() to access it. Because perf an ftrace have no "module"
selection, there will be no in-tree module that would use that interface,
thus LTTng is out of luck.

This forces LTTng to come up with a different solution for everything that
perf and ftrace do.

This is a huge catch-22 for LTTng. You want unification before LTTng can
become part of the kernel. But because Christoph and Greg refuse to allow
LTTng to have access to anything that perf and ftrace use, there's no path
for LTTng to share the code.

This discussion is going full circle:

1. Mathieu helps with new infrastructure that it could share with perf and
   ftrace.

2. We ask to add EXPORT_SYMBOL_GPL() so that LTTng can share this code.

3. It is denied because LTTng is out of tree, and we are told that if LTTng
   wants to use this it must first be in tree.

4. We send an RFC on how to get LTTng in tree.

5. We are told that LTTng can only get in tree through incremental steps.

6. The only way to do incremental steps is for LTTng to have access to the
   same functions as perf and ftrace have, but is not allow to because it's
   out of tree!

When I said being out of tree makes it much more difficult to merge code,
this is the reason why.

Breaking up LTTng is a huge task to do it in incremental states. If it can't
use in kernel infrastructure while it is out of tree, it will mean the out
of tree LTTng cannot use the code being submitted. Thus, the code going
upstream will be unused or will be a non-functional LTTng. If it takes several
years to complete, it is forcing Mathieu to have a broken LTTng for several
years before it is allowed to use code upstream.

So which is it?

Either LTTng can use the same infrastructure as perf and ftrace, meaning it
requires EXPORT_SYMBOL_GPL() on those interfaces, or it gets pulled in as
is, so that it can incrementally start to share the code.

-- Steve

Powered by blists - more mailing lists