[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <Y3eJ8GiGnEvVd8/N@macondo>
Date: Fri, 18 Nov 2022 10:34:40 -0300
From: Rafael Mendonca <rafaelmendsr@...il.com>
To: Steven Rostedt <rostedt@...dmis.org>
Cc: Masami Hiramatsu <mhiramat@...nel.org>,
"Tzvetomir Stoyanov (VMware)" <tz.stoyanov@...il.com>,
linux-kernel@...r.kernel.org, linux-trace-kernel@...r.kernel.org,
Tom Zanussi <zanussi@...nel.org>
Subject: Re: [PATCH] tracing/eprobe: Update cond flag before enabling trigger
On Fri, Nov 18, 2022 at 09:40:09AM -0300, Rafael Mendonca wrote:
> On Thu, Nov 17, 2022 at 09:31:09PM -0500, Steven Rostedt wrote:
> > On Thu, 17 Nov 2022 21:17:26 -0500
> > Steven Rostedt <rostedt@...dmis.org> wrote:
> >
> > > On Wed, 16 Nov 2022 16:25:51 -0300
> > > Rafael Mendonca <rafaelmendsr@...il.com> wrote:
> > >
> > > > That happens because enable_eprobe() will eventually trigger the
> > > > kmem/mm_page_alloc trace event:
> > > >
> > > > - enable_eprobe [trace_eprobe.c]
> > > > - trace_event_trigger_enable_disable [trace_events_trigger.c]
> > > > - trace_event_enable_disable [trace_events.c]
> > > > - __ftrace_event_enable_disable [trace_events.c]
> > > > - trace_buffered_event_enable [trace.c]
> > > > - alloc_pages_node [gfp.h]
> > > > ...
> > > > - __alloc_pages [page_alloc.c]
> > > > - trace_mm_page_alloc // eprobe event file without TRIGGER_COND bit set
> > > >
> > > > By the time kmem/mm_page_alloc trace event is hit, the eprobe event file
> > > > does not have the TRIGGER_COND flag set yet, which causes the eprobe's
> > > > trigger to be invoked (through the trace_trigger_soft_disabled() path)
> > > > without a trace record, causing a NULL pointer dereference when fetching
> > > > the event fields.
> > > >
> > > > Fix this by setting the cond flag beforehand when enabling the eprobe's
> > > > trigger.
> > > >
> > > > Fixes: 7491e2c44278 ("tracing: Add a probe that attaches to trace events")
> > > > Signed-off-by: Rafael Mendonca <rafaelmendsr@...il.com>
> > > > ---
> > >
> > > Thanks for the report, but I'm worried that this isn't enough because of
> > > how memory ordering can happen on different architectures. That is, just
> > > because you switch the order of updates, doesn't mean that the architecture
> > > will honor it.
> > >
> > > I don't want to add memory barriers in the fast path, but instead we can
> > > simply check if rec is NULL in the handler.
> > >
> > > So basically:
> > >
> > >
> > > static void eprobe_trigger_func(struct event_trigger_data *data,
> > > struct trace_buffer *buffer, void *rec,
> > > struct ring_buffer_event *rbe)
> > > {
> > > struct eprobe_data *edata = data->private_data;
> > >
> > > if (!rec)
> > > return;
> > >
> > > __eprobe_trace_func(edata, rec);
> > > }
> > >
> > > And this should be documented.
> > >
> >
> > In fact, does this fix it for you?
>
> It does. I found this while enabling eprobe for all events in my setup.
> Doing the same test wih the proposed patch it did not trigger any issue.
> Thanks.
It did not trigger the NULL pointer issue to be more specific. When
creating event probe for all events I was unable to create any event for
the xhci-hcd system:
root@...alhost:/sys/kernel/tracing# echo 'e xhci-hcd/xhci_add_endpoint' > dynamic_events
-bash: echo: write error: Invalid argument
Debugging the issue it seems that the problem is in the is_good_name()
check, which returns false for "xhci-hcd". Should we sanitize it by
converting '-' into '_'?
>
> >
> > I'm going to take this patch and reference you as a reported-by, as I have
> > a lot of urgent code that needs to got upstream, and I need to start
> > testing it.
> >
> > Thanks!
> >
> > -- Steve
> >
> > diff --git a/kernel/trace/trace_eprobe.c b/kernel/trace/trace_eprobe.c
> > index 5dd0617e5df6..6b31b74954d9 100644
> > --- a/kernel/trace/trace_eprobe.c
> > +++ b/kernel/trace/trace_eprobe.c
> > @@ -563,6 +563,9 @@ static void eprobe_trigger_func(struct event_trigger_data *data,
> > {
> > struct eprobe_data *edata = data->private_data;
> >
> > + if (!rec)
> > + return;
> > +
> > __eprobe_trace_func(edata, rec);
> > }
> >
> >
Powered by blists - more mailing lists