Date:   Wed, 21 Apr 2021 23:37:47 +0200
From:   Jiri Olsa <jolsa@...hat.com>
To:     Steven Rostedt <rostedt@...dmis.org>
Cc:     Alexei Starovoitov <alexei.starovoitov@...il.com>,
        Masami Hiramatsu <mhiramat@...nel.org>,
        Andrii Nakryiko <andrii.nakryiko@...il.com>,
        Jiri Olsa <jolsa@...nel.org>,
        Alexei Starovoitov <ast@...nel.org>,
        Daniel Borkmann <daniel@...earbox.net>,
        Andrii Nakryiko <andriin@...com>,
        Networking <netdev@...r.kernel.org>, bpf <bpf@...r.kernel.org>,
        Martin KaFai Lau <kafai@...com>,
        Song Liu <songliubraving@...com>, Yonghong Song <yhs@...com>,
        John Fastabend <john.fastabend@...il.com>,
        KP Singh <kpsingh@...omium.org>, Daniel Xu <dxu@...uu.xyz>,
        Jesper Brouer <jbrouer@...hat.com>,
        Toke Høiland-Jørgensen <toke@...hat.com>,
        Viktor Malik <vmalik@...hat.com>
Subject: Re: [PATCHv2 RFC bpf-next 0/7] bpf: Add support for ftrace probe

On Wed, Apr 21, 2021 at 10:05:41AM -0400, Steven Rostedt wrote:
> On Wed, 21 Apr 2021 15:40:37 +0200
> Jiri Olsa <jolsa@...hat.com> wrote:
> 
> > ok, I understand why this would be the best solution for calling
> > the program from multiple probes
> > 
> > I think it's the 'attach' layer which is the source of problems
> > 
> > currently there is ftrace's fgraph_ops support that allows fast mass
> > attach and calls callbacks for functions entry and exit:
> >   https://lore.kernel.org/lkml/20190525031633.811342628@goodmis.org/
> > 
> > these callbacks get ip/parent_ip and can get pt_regs (that's not
> > implemented at the moment)
> > 
> > but that gets us to the situation of having full pt_regs on both
> > entry/exit callbacks that you described above and want to avoid,
> > but I think it's the price for having this on top of generic
> > tracing layer
> > 
> > the way ftrace's fgraph_ops is implemented, I'm not sure it can
> > be as fast as current bpf entry/exit trampoline
> 
> Note, the above mentioned code was an attempt to consolidate the code that
> does the "highjacking" of the return pointer in order to record the
> return of a function. At the time there was only kretprobes and function
> graph tracing. Now bpf has another version. That means there's three
> utilities that record the exit of the function.
> 
> What we need is a single method that works for all three utilities. And I'm
> perfectly fine with a rewrite of function graph tracer to do that. The one
> problem is that function graph and kretprobes works for pretty much all the
> architectures now, and whatever we decide to do, we can't break those
> architectures.
> 
> One way is to have an abstract layer that allows function graph and
> kretprobes to work with the old implementation as well as, depending on a
> config set, a new implementation that also supports bpf trampolines.
> 
> > 
> > but to better understand the pain points I think I'll try to implement
> > the 'mass trampolines' call to the bpf program you described above and
> > attach it for now to fgraph_ops callbacks
> 
> One thing that ftrace gives you is a way to have each function call its own
> trampoline, then depending on what is attached, each one can have multiple
> implementations.

but that would cut off other tracers for the function, right?

AFAICT it's used only when there's single ftrace_ops registered
for the probe and if there are more ftrace_ops, ops->trampoline
is replaced by the generic one and ftrace will call ops->func
instead, right?

if we would not care about sharing function by multiple tracers,
we could make special 'exclusive' trampoline that would require
that no other tracer is (or will be) registered for the function
while the tracer is registered

then we could run BPF trampolines directly without 'direct' API
and use ftrace for mass attach

that is if we don't care about other tracers for the function,
which I guess was concern when the 'direct' API was introduced

jirka

> 
> One thing that needs to be fixed is the direct trampoline and function
> graph and kretprobes. As the direct trampoline will break both of them,
> with the bpf implementation to trace after it.
> 
> I would be interested in what a mass generic trampoline would look like, if
> it had to deal with handling functions with 1 parameter and one with 12
> parameters. From this thread, I was told it can currently only handle 6
> parameters on x86_64. Not sure how it works on x86_32.
> 
> > 
> > perhaps this is a good topic to discuss in one of the Thursday's BPF mtg?
> 
> I'm unaware of these meetings.
> 
> 
> -- Steve
> 

Powered by blists - more mailing lists