linux-kernel - [tip:perf/core] perf/core: Introduce PMU_EV_CAP_READ_ACTIVE

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <tip-d6a2f9035bfc27d0e9d78b13635dda9fb017ac01@git.kernel.org>
Date:	Thu, 18 Aug 2016 03:54:23 -0700
From:	tip-bot for David Carrillo-Cisneros <tipbot@...or.com>
To:	linux-tip-commits@...r.kernel.org
Cc:	vincent.weaver@...ne.edu, torvalds@...ux-foundation.org,
	eranian@...gle.com, davidcc@...gle.com, peterz@...radead.org,
	linux-kernel@...r.kernel.org, alexander.shishkin@...ux.intel.com,
	pjt@...gle.com, hpa@...or.com, mingo@...nel.org, jolsa@...hat.com,
	vegard.nossum@...il.com, acme@...hat.com, tglx@...utronix.de,
	kan.liang@...el.com
Subject: [tip:perf/core] perf/core: Introduce PMU_EV_CAP_READ_ACTIVE_PKG

Commit-ID:  d6a2f9035bfc27d0e9d78b13635dda9fb017ac01
Gitweb:     http://git.kernel.org/tip/d6a2f9035bfc27d0e9d78b13635dda9fb017ac01
Author:     David Carrillo-Cisneros <davidcc@...gle.com>
AuthorDate: Wed, 17 Aug 2016 13:55:06 -0700
Committer:  Ingo Molnar <mingo@...nel.org>
CommitDate: Thu, 18 Aug 2016 10:53:59 +0200

perf/core: Introduce PMU_EV_CAP_READ_ACTIVE_PKG

Introduce the flag PMU_EV_CAP_READ_ACTIVE_PKG, useful for uncore events,
that allows a PMU to signal the generic perf code that an event is readable
in the current CPU if the event is active in a CPU in the same package as
the current CPU.

This is an optimization that avoids a unnecessary IPI for the common case
where uncore events are run and read in the same package but in
different CPUs.

As an example, the IPI removal speeds up perf_read() in my Haswell system
as follows:

  - For event UNC_C_LLC_LOOKUP: From 260 us to 31 us.
  - For event RAPL's power/energy-cores/: From to 255 us to 27 us.

For the optimization to work, all events in the group must have it
(similarly to PERF_EV_CAP_SOFTWARE).

Signed-off-by: David Carrillo-Cisneros <davidcc@...gle.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@...radead.org>
Cc: Alexander Shishkin <alexander.shishkin@...ux.intel.com>
Cc: Arnaldo Carvalho de Melo <acme@...hat.com>
Cc: David Carrillo-Cisneros <davidcc@...gle.com>
Cc: Jiri Olsa <jolsa@...hat.com>
Cc: Kan Liang <kan.liang@...el.com>
Cc: Linus Torvalds <torvalds@...ux-foundation.org>
Cc: Paul Turner <pjt@...gle.com>
Cc: Peter Zijlstra <peterz@...radead.org>
Cc: Stephane Eranian <eranian@...gle.com>
Cc: Thomas Gleixner <tglx@...utronix.de>
Cc: Vegard Nossum <vegard.nossum@...il.com>
Cc: Vince Weaver <vincent.weaver@...ne.edu>
Link: http://lkml.kernel.org/r/1471467307-61171-4-git-send-email-davidcc@google.com
Signed-off-by: Ingo Molnar <mingo@...nel.org>
---
 include/linux/perf_event.h |  3 +++
 kernel/events/core.c       | 25 +++++++++++++++++++++++--
 2 files changed, 26 insertions(+), 2 deletions(-)

diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
index 6f7459f..5c53625 100644
--- a/include/linux/perf_event.h
+++ b/include/linux/perf_event.h
@@ -514,8 +514,11 @@ typedef void (*perf_overflow_handler_t)(struct perf_event *,
  * Event capabilities. For event_caps and groups caps.
  *
  * PERF_EV_CAP_SOFTWARE: Is a software event.
+ * PERF_EV_CAP_READ_ACTIVE_PKG: A CPU event (or cgroup event) that can be read
+ * from any CPU in the package where it is active.
  */
 #define PERF_EV_CAP_SOFTWARE		BIT(0)
+#define PERF_EV_CAP_READ_ACTIVE_PKG	BIT(1)
 
 #define SWEVENT_HLIST_BITS		8
 #define SWEVENT_HLIST_SIZE		(1 << SWEVENT_HLIST_BITS)
diff --git a/kernel/events/core.c b/kernel/events/core.c
index 8c42a5a..3f07e6c 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -3424,6 +3424,22 @@ struct perf_read_data {
 	int ret;
 };
 
+static int find_cpu_to_read(struct perf_event *event, int local_cpu)
+{
+	int event_cpu = event->oncpu;
+	u16 local_pkg, event_pkg;
+
+	if (event->group_caps & PERF_EV_CAP_READ_ACTIVE_PKG) {
+		event_pkg =  topology_physical_package_id(event_cpu);
+		local_pkg =  topology_physical_package_id(local_cpu);
+
+		if (event_pkg == local_pkg)
+			return local_cpu;
+	}
+
+	return event_cpu;
+}
+
 /*
  * Cross CPU call to read the hardware event
  */
@@ -3545,7 +3561,7 @@ u64 perf_event_read_local(struct perf_event *event)
 
 static int perf_event_read(struct perf_event *event, bool group)
 {
-	int ret = 0;
+	int ret = 0, cpu_to_read, local_cpu;
 
 	/*
 	 * If event is enabled and currently active on a CPU, update the
@@ -3557,7 +3573,12 @@ static int perf_event_read(struct perf_event *event, bool group)
 			.group = group,
 			.ret = 0,
 		};
-		ret = smp_call_function_single(event->oncpu, __perf_event_read, &data, 1);
+
+		local_cpu = get_cpu();
+		cpu_to_read = find_cpu_to_read(event, local_cpu);
+		put_cpu();
+
+		ret = smp_call_function_single(cpu_to_read, __perf_event_read, &data, 1);
 		/* The event must have been read from an online CPU: */
 		WARN_ON_ONCE(ret);
 		ret = ret ? : data.ret;