linux-kernel - Re: malloc() tracing in perf?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20090806115527.GE18768@redhat.com>
Date:	Thu, 6 Aug 2009 07:55:27 -0400
From:	"Frank Ch. Eigler" <fche@...hat.com>
To:	Peter Zijlstra <a.p.zijlstra@...llo.nl>
Cc:	Pekka Enberg <penberg@...helsinki.fi>, Ingo Molnar <mingo@...e.hu>,
	acme <acme@...hat.com>, Steven Rostedt <rostedt@...dmis.org>,
	Frédéric Weisbecker <fweisbec@...il.com>,
	Eduard-Gabriel Munteanu <eduard.munteanu@...ux360.ro>,
	roland <roland@...hat.com>,
	Christoph Hellwig <hch@...radead.org>,
	Masami Hiramatsu <mhiramat@...hat.com>,
	Mathieu Desnoyers <mathieu.desnoyers@...ymtl.ca>,
	linux-kernel@...r.kernel.org
Subject: Re: malloc() tracing in perf?

Hi -

On Thu, Aug 06, 2009 at 01:28:14PM +0200, Peter Zijlstra wrote:
> [...]
> > That work is ongoing, and being discussed on utrace-devel@...hat.com,
> > since it is a prerequisite. 
> 
> Still hiding the discussion and the design never helped anybody.

I guess "hiding" is a matter of opinion.

> > While these deliberations are ongoing, you can use systemtap.  Probing
> > random places in userspace is about as casual as probing the kernel:
> 
> Right, but that still doesn't tell us anything on how you're doing that,
> does it?

Since you asked...  the probe process("/lib64/libc.so.6") points
systemtap to a shared library file, whose symbol table & debug data
gives us information about what functions & parameters are available.
Among other things, we record a shared-library base-relative address
for the function.

At run time, we monitor the entire system (or just a given process if
-x PID/-c CMD was specified) to see when that shared library gets
loaded.  (This in turn is done with a utrace-based hook of the mmap
syscall - see "task_finder" in our sources).  When it's loaded, we can
finally compute a run-time address for the probed malloc/free
functions.  At this point we insert a uprobe (using the uprobes module
API, which also relies on utrace.)  Should the shared library become
unmapped, or the application exit, the uprobe will be removed.

When/if the uprobe eventually hits, we run the compiled probe handler.
As usual, it can do lots of stuff beside just simple tracing, but in
the case of that statement (println ($$parms ...))  that's all we do.
$$parms was expanded at compilation time to the list of variables, and
instructions about how to fetch their values at run time.  So now
those instructions are run, their values formatted, and the tracing
message sent to our buffer.

This overview skims over issues related to return probes, tracing
buffer manipulations, and much other stuff.

All the code for this is hidden in plain sight in every systemtap
release, so please feel free to refer to that and/or ask more detailed
questions.

- FChE
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/