Date:	Mon, 18 Sep 2006 02:07:03 +0200
From:	Ingo Molnar <mingo@...e.hu>
To:	Mathieu Desnoyers <mathieu.desnoyers@...ymtl.ca>
Cc:	Paul Mundt <lethal@...ux-sh.org>,
	Karim Yaghmour <karim@...rsys.com>,
	linux-kernel <linux-kernel@...r.kernel.org>,
	Ingo Molnar <mingo@...hat.com>, Jes Sorensen <jes@....com>,
	Andrew Morton <akpm@...l.org>,
	Roman Zippel <zippel@...ux-m68k.org>,
	Tom Zanussi <zanussi@...ibm.com>,
	Richard J Moore <richardj_moore@...ibm.com>,
	"Frank Ch. Eigler" <fche@...hat.com>,
	Michel Dagenais <michel.dagenais@...ymtl.ca>,
	Christoph Hellwig <hch@...radead.org>,
	Greg Kroah-Hartman <gregkh@...e.de>,
	Thomas Gleixner <tglx@...utronix.de>,
	William Cohen <wcohen@...hat.com>,
	"Martin J. Bligh" <mbligh@...igh.org>
Subject: Re: tracepoint maintainance models


* Mathieu Desnoyers <mathieu.desnoyers@...ymtl.ca> wrote:

> * Ingo Molnar (mingo@...e.hu) wrote:
> >   - model #2: we could have the least intrusive markers in the main
> >     kernel source, while the more intrusive ones would still be in the
> >     upstream kernel, but in scripts/instrumentation/.
> > 
> 
> Please define : marker intrusiveness. I think that this is not a sole 
> concept. First, I think we have to look at intrusiveness under three 
> different angles :
> 
> - Visual intrusiveness (hurts visually in the code)
> - Compiled-in, but inactive intrusiveness
>   - Modifies compiler optimisations when the marker is compiled in but no
>     tracing is active.
>   - Wastes a few cycles because it adds NOPs, jump, etc in a critical path
>     when tracing is not active.
> - Active tracing intrusiveness
>   - Wastes too many cycles in a critical path when tracing is active.

as the primary factor i'd add:

  - Maintainance intrusiveness

but yes, agreed - with that addition this is a good summary of the 
intrusiveness factors.

> The problem is that a static marker will speed up the active tracing 
> while a dynamic probe will speed up the case where tracing is 
> inactive. The problem is that the dynamic probe cost can get so big 
> that it modifies the traced system often more than acceptable. [...]

do you base this opinion of yours on the kprobes+LTT experiment you did 
yesterday? If yes then would it be possible for you to try the 3 patches 
that i sent, and re-measure the impact of kprobes? The kprobes overhead 
should go down a bit, it would be interesting to see by how much.

Also, when forming your opinion do you consider djprobes - which in 
essence inserts a function call (and not an INT3) into the probed code?

> [...] Under this angle, I would be tempted to say that the most 
> intrusive instrumentation should be helped by marker, which means 
> accepting a very small performance impact (NOPs on modern CPUs are 
> quite fast) when tracing is not active in order to enable fast tracing 
> of some very high event rate kernel code paths.

i'm not fundamentally worried about the runtime impact of static probes, 
as long as the impact is unmeasurable. I'm more worried about their 
maintainance impact - so i want the option to move them to a dynamic 
script, if the tracepoint is for example not frequent. (but being a 
perfectionist i cannot completely forget about their runtime overhead 
either)

> >   - model #3: we could have the 'hardest' markups in the source, and the 
> >     'easy' ones as dynamic markups in scripts/instrumentation/.
> > 
> By "hardest", do you mean : where the data that is to be extracted is 
> not easily available due to compiler optimisations ?

yeah - hard in the sense of dynamic probing.

> > i agree, as long as it's lightweight markers for _dynamic tracers_, 
> > so that we keep our options open - as per the arguments above.
> 
> But I also think that a marker mechanism should not only mark the code 
> location where the instrumentation is to be made, but also the 
> information the probe is interested into (provide compile-time data 
> type verification and address at runtime). Doing otherwise would limit 
> what could be provided to static markup users.

yeah. If you look at the API suggestions i made, they are such. There 
can be differences though to 'static tracepoints used by static 
tracers': for example there's no need to 'mark' a static variable, 
because dynamic tracers have access to it - while a static tracer would 
have to pass it into its trace-event function call.

	Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists