Message-ID: <BANLkTin-05pi94_qDw8Qkn2jCpv_V9j6sw@mail.gmail.com>
Date:	Fri, 8 Apr 2011 13:27:00 -0700
From:	Justin TerAvest <teravest@...gle.com>
To:	Michael Rubin <mrubin@...gle.com>
Cc:	Peter Zijlstra <a.p.zijlstra@...llo.nl>,
	Frederic Weisbecker <fweisbec@...il.com>,
	Steven Rostedt <rostedt@...dmis.org>,
	David Sharp <dhsharp@...gle.com>,
	Vaibhav Nagarnaik <vnagarnaik@...gle.com>,
	Paul Menage <menage@...gle.com>,
	Li Zefan <lizf@...fujitsu.com>,
	Stephane Eranian <eranian@...gle.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Ken Chen <kenchen@...gle.com>, linux-kernel@...r.kernel.org,
	containers@...ts.linux-foundation.org,
	Thomas Gleixner <tglx@...utronix.de>,
	Ingo Molnar <mingo@...e.hu>,
	Arnaldo Carvalho de Melo <acme@...hat.com>
Subject: Re: [RFC] tracing: Adding cgroup aware tracing functionality

On Fri, Apr 8, 2011 at 11:32 AM, Michael Rubin <mrubin@...gle.com> wrote:
> Minor history, Google wrote ktrace which we started to push upstream
> and then jumped on the ftrace train. ftrace was a very good drop in
> replacement for us. ftrace has a lot of features ktrace did not. We
> use perf also in other contexts.
>
> We have devoted a lot of time and energy from many engineers outside
> of kernel to use ftrace. It's not just the kernel code, but we have
> built a lot of sw on ftrace as we examine systems in the cloud. It was
> not easy to make this migration since it had a lot of user impact. It
> was less the low level API but the fact that the tracepoints semantics
> had changed. Google is in the process of no longer using ktrace.
>
> Requirements for tracing for us seem to be:
> 1) Around 250ns latencies when enabled

It's worth pointing out that this is a goal for us on ftrace.

We were able to do this well with ktrace, and have devoted effort to
making ftrace faster, but we're not at 250ns yet. We know that is
where we want to be.

> 2) Small events - We have worked on shrinking the ftrace event size.
> Since we are constrained on the amount of memory we can use in the
> ring buffer, every byte counts. A nice sized event is on the order of
> <= 8 bytes. Some of the 2.6.34 ftrace events like irq_enter and exit
> were too big around 20 bytes. sched_switch is around 60 bytes.
>
> I am not sure that perf meets these requirements. We need to use
> ftrace for tracing _now_.  Google also needs to focus on improving
> ftrace upstream so we can contribute our changes to the community. We
> are submitting patches and working with upstream to improve ftrace.
> Currently we depend on ftrace and being around for a long time.
>
> Fredric wrote:" Nonetheless you can't be such a significant
> user/developer of the upstream kernel tracing and at the same time
> ignore some key problems of the actual big picture of it."
>
> I see less of a big picture issue and more of an awkward two pictures.
> From my view it's a question of whether ftrace should continue to be
> supported and improved or whether perf will do everything as well and
> when. We depend on ftrace today and as such we have invested our time
> there.
>
> mrubin
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@...r.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists