Message-ID: <20211116104631.195cbd0b@eldfell>
Date:   Tue, 16 Nov 2021 10:46:31 +0200
From:   Pekka Paalanen <ppaalanen@...il.com>
To:     Jason Baron <jbaron@...mai.com>
Cc:     Vincent Whitchurch <vincent.whitchurch@...s.com>,
        Jim Cromie <jim.cromie@...il.com>, quic_saipraka@...cinc.com,
        catalin.marinas@....com, dri-devel@...ts.freedesktop.org,
        will@...nel.org, maz@...nel.org, amd-gfx@...ts.freedesktop.org,
        mingo@...hat.com, daniel.vetter@...ll.ch, arnd@...db.de,
        linux-arm-msm@...r.kernel.org, intel-gfx@...ts.freedesktop.org,
        rostedt@...dmis.org, seanpaul@...omium.org,
        intel-gvt-dev@...ts.freedesktop.org,
        linux-arm-kernel@...ts.infradead.org, sean@...rly.run,
        gregkh@...uxfoundation.org, linux-kernel@...r.kernel.org,
        quic_psodagud@...cinc.com, mathieu.desnoyers@...icios.com
Subject: Re: [PATCH v10 08/10] dyndbg: add print-to-tracefs, selftest with
 it - RFC

On Fri, 12 Nov 2021 10:08:41 -0500
Jason Baron <jbaron@...mai.com> wrote:

> On 11/12/21 6:49 AM, Vincent Whitchurch wrote:
> > On Thu, Nov 11, 2021 at 03:02:04PM -0700, Jim Cromie wrote:  
> >> Sean Paul proposed, in:
> >> https://urldefense.com/v3/__https://patchwork.freedesktop.org/series/78133/__;!!GjvTz_vk!HcKnMRByYkIdyF1apqQjlN5aBIomzJR1an3YWXM6KXs0EftVMQdrewRA8Dki4A$ 
> >> drm/trace: Mirror DRM debug logs to tracefs
> >>
> >> His patchset's objective is to be able to independently steer some of
> >> the drm.debug stream to an alternate tracing destination, by splitting
> >> drm_debug_enabled() into syslog & trace flavors, and enabling them
> >> separately.  2 advantages were identified:
> >>
> >> 1- syslog is heavyweight, tracefs is much lighter
> >> 2- separate selection of enabled categories means less traffic
> >>
> >> Dynamic-Debug can do 2nd exceedingly well:
> >>
> >> A- all work is behind jump-label's NOOP, zero off cost.
> >> B- exact site selectivity, precisely the useful traffic.
> >>    can tailor enabled set interactively, at shell.
> >>
> >> Since the tracefs interface is effective for drm (the threads suggest
> >> so), adding that interface to dynamic-debug has real potential for
> >> everyone including drm.
> >>
> >> if CONFIG_TRACING:
> >>
> >> Grab Sean's trace_init/cleanup code, use it to provide tracefs
> >> available by default to all pr_debugs.  This will likely need some
> >> further per-module treatment; perhaps something reflecting hierarchy
> >> of module,file,function,line, maybe with a tuned flattening.
> >>
> >> endif CONFIG_TRACING
> >>
> >> Add a new +T flag to enable tracing, independent of +p, and add and
> >> use 3 macros: dyndbg_site_is_enabled/logging/tracing(), to encapsulate
> >> the flag checks.  Existing code treats T like other flags.  
> > 
> > I posted a patchset a while ago to do something very similar, but that
> > got stalled for some reason and I unfortunately didn't follow it up:
> > 
> >  https://urldefense.com/v3/__https://lore.kernel.org/lkml/20200825153338.17061-1-vincent.whitchurch@axis.com/__;!!GjvTz_vk!HcKnMRByYkIdyF1apqQjlN5aBIomzJR1an3YWXM6KXs0EftVMQdrewRGytKHPg$ 
> > 
> > A key difference between that patchset and this patch (besides that
> > small fact that I used +x instead of +T) was that my patchset allowed
> > the dyndbg trace to be emitted to the main buffer and did not force them
> > to be in an instance-specific buffer.  
> 
> Yes, I agree I'd prefer that we print here to the 'main' buffer - it
> seems to keep things simpler and easier to combine the output from
> different sources as you mentioned.

Hi,

I'm not quite sure I understand this discussion, but I would like to
remind you all of what Sean's original work is about:

Userspace configures DRM tracing into a flight recorder buffer (I guess
this is what you refer to "instance-specific buffer").

Userspace runs happily for months, and then hits a problem: a failure
in the DRM sub-system most likely, e.g. an ioctl that should never
fail, failed. Userspace handles that failure by dumping the flight
recorder buffer into a file and saving or sending a bug report. The
flight recorder contents give a log of all relevant DRM in-kernel
actions leading to the unexpected failure to help developers debug it.

I don't mind if one can additionally send the flight recorder stream to
the main buffer, but I do want the separate flight recorder buffer to
be an option so that a) unrelated things cannot flood the interesting
bits out of it, and b) the scope of collected information is relevant.

The very reason for this work is problems that are very difficult to
reproduce in practice, either because the problem itself is triggered
very rarely and randomly, or because the end users of the system have
either no knowledge or no access to reconfigure debug logging and then
reproduce the problem with good debug logs.

Thank you very much for pushing this work forward!


Thanks,
pq

> 
> Thanks,
> 
> -Jason
> 
> > 
> > That feature is quite important at least for my use case since I often
> > use dyndbg combined with function tracing, and the latter doesn't work
> > on non-main instances according to Documentation/trace/ftrace.rst.
> > 
> > For example, here's a random example of a bootargs from one of my recent
> > debugging sessions:
> > 
> >  trace_event=printk:* ftrace_filter=_mmc*,mmc*,sd*,dw_mci*,mci*
> >  ftrace=function trace_buf_size=20M dyndbg="file drivers/mmc/* +x"
> >   


Content of type "application/pgp-signature" skipped

Powered by blists - more mailing lists