linux-kernel - perf_event self-monitoring overhead regression

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <alpine.DEB.2.00.1111300011260.7107@cl320.eecs.utk.edu>
Date:	Wed, 30 Nov 2011 00:20:04 -0500
From:	Vince Weaver <vweaver1@...s.utk.edu>
To:	Ingo Molnar <mingo@...e.hu>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	<a.p.zijlstra@...llo.nl>, Paul Mackerras <paulus@...ba.org>,
	Arnaldo Carvalho de Melo <acme@...stprotocols.net>,
	Stephane Eranian <eranian@...il.com>
Subject: perf_event self-monitoring overhead regression

Hello

I've been tracking a performance regression with self-monitoring and 
perf_event.

For a simple start/stop/read test, the overhead has increased about 10%
from the 2.6.32 kernel to 3.1.  (this has been measured on a variety
of x86_64 machines).

This is as measured with the POSIX clock_gettime(CLOCK_REALTIME,&time)
calls.  Potentially the issue is with this and not with perf_events.
As you can imagine it is hard to measure the performance of the perf_event
interface since you can't invoke perf_event on it.

In any case, I was trying to bisect some of these performance issues.  
There was another jump in overhead between 3.0 and 3.1, so I tried there.
I had a bisectable test case, but after a tedious day-long bisect run the 
problem bisected down to

   commit 2d86a3f04e345b03d5e429bfe14985ce26bff4dc
   Merge: 3960ef3 5ddac6b
   Author: Linus Torvalds <torvalds@...ux-foundation.org>
   Date:   Tue Jul 26 17:13:04 2011 -0700

    Merge branch 'next/board' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/l

Which seems unlikely.  My git skills really aren't enough to try to figure
out why an ARM board merge would affect the overhead of the perf_event
syscalls on x86_64.

Is there a better way for trying to track down performance regressions 
like this?

Thanks,

Vince
vweaver1@...s.utk.edu

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/