Date:   Thu,  3 Nov 2016 12:52:29 +0530
From:   Hemant Kumar <hemant@...ux.vnet.ibm.com>
To:     linux-kernel@...r.kernel.org
Cc:     maddy@...ux.vnet.ibm.com, mpe@...erman.id.au,
        benh@...nel.crashing.org, paulus@...ba.org, anton@...ba.org,
        sukadev@...ux.vnet.ibm.com, mikey@...ling.org,
        stewart@...ux.vnet.ibm.com, eranian@...gle.com,
        Hemant Kumar <hemant@...ux.vnet.ibm.com>
Subject: [PATCH 0/6] IMA Instrumentation support

Powerpc has In-Memory-Accumulation (IMA) infrastructure which contains
various Performance Monitoring Units (PMUs) at Nest level (these are
on-chip but off-core). These Nest PMU counters are handled by a Nest
IMA microcode. This microcode runs in the OCC (On-Chip Controller)
complex and its purpose is to program the nest counters, collect the
counter data and move the counter data to memory. 

The IMA infrastructure encapsulates nest (per-chip), core and thread
level counters. While the nest IMA PMUs are handled by the nest IMA
microcode, the core and thread level PMUs are handled by the Core-HPMC
engine. This patchset enables the nest IMA PMUs and is based on the
initial work done by Madhavan Srinivasan.
"Nest Instrumentation Support" : https://lists.ozlabs.org/pipermail/linuxppc-dev/2015-August/132078.html

Nest events:
Per-chip nest instrumentation provides various per-chip metrics
such as memory, powerbus, Xlink and Alink bandwidth.

PMU Events' Information:
OPAL obtains the Nest PMU and event information from the IMA Catalog
and passes on to the kernel via the device tree. The events' information
contains :
 - Event name
 - Event Offset
 - Event description
and, maybe :
 - Event scale
 - Event unit

The event offset in the memory is where the counter data gets
accumulated.

The OPAL-side patches are posted upstream :
https://lists.ozlabs.org/pipermail/skiboot/2016-October/005235.html

The kernel discovers the IMA counters information in the device tree
at the "ima-counters" device node which has a compatible field
"ibm,opal-in-memory-counters".

Parsing of the Events' information:
To parse the IMA PMUs and events information, the kernel has to
discover the "ima-counters" node and walk through the pmu and event
nodes.

Here is an excerpt of the dt showing the ima-counters and mcs node:
/dts-v1/;

[...]
        ima-counters {                                                   
                ima-nest-offset = <0x320000>;
                compatible = "ibm,opal-in-memory-counters";
                ima-nest-size = <0x30000>;
                #address-cells = <0x1>;
                #size-cells = <0x1>;
                phandle = <0x10000238>;
                version-id = [00];

		mcs {                                                    
                        compatible = "ibm,ima-counters-chip";
                        ranges;
                        #address-cells = <0x1>;
                        #size-cells = <0x1>;
                        phandle = <0x10000279>;

                        PM_MCS_UP_128B_DATA_XFER_MC0@528 {
                       		desc = "Total Read Bandwidth seen on both MCS of MC0";
                                phandle = <0x1000028c>;
                                reg = <0x118 0x8>;
				scale = "1.2207e-4";
				unit = "MiB";
                        };
[...]

>From the device tree, the kernel parses the PMUs and their events'
information.

After parsing the nest IMA PMUs and their events, the PMUs and their
attributes are registered in the kernel.

Example Usage :
 # perf list

  [...]
  nest_alink/PM_ALINK0_IN_DATA_CYC/                  [Kernel PMU event]
  nest_alink/PM_ALINK0_IN_DATA_CYC_LAST_SAMPLE/      [Kernel PMU event]
  nest_alink/PM_ALINK0_IN_IDL_CYC/                   [Kernel PMU event]
  [...]
  nest_mcs/PM_MCS_DOWN_128B_DATA_XFER_MC0/           [Kernel PMU event]
  nest_mcs/PM_MCS_DOWN_128B_DATA_XFER_MC0_LAST_SAMPLE/ [Kernel PMU event]
  nest_mcs/PM_MCS_DOWN_128B_DATA_XFER_MC1/           [Kernel PMU event]
  [...]
  nest_phb/PM_PHB_ANY_DMA_RCV_PHB0/                  [Kernel PMU event]
  nest_phb/PM_PHB_ANY_DMA_RCV_PHB0_LAST_SAMPLE/      [Kernel PMU event]
  nest_phb/PM_PHB_ANY_DMA_RCV_PHB1/                  [Kernel PMU event]
  [...]

 # perf stat -e "nest_mcs/PM_MCS_DOWN_128B_DATA_XFER_MC2/" -e "nest_mcs/PM_MCS_UP_128B_DATA_XFER_MC2/" -a --per-socket

TODOs:
 - Add support for Core IMA.
 - Add support for thread IMA.

Comments/feedback/suggestions are welcome.

Cc: Madhavan Srinivasan <maddy@...ux.vnet.ibm.com>
Cc: Michael Ellerman <mpe@...erman.id.au>
Cc: Benjamin Herrenschmidt <benh@...nel.crashing.org>
Cc: Paul Mackerras <paulus@...ba.org>
Cc: Anton Blanchard <anton@...ba.org>
Cc: Sukadev Bhattiprolu <sukadev@...ux.vnet.ibm.com>
Cc: Michael Neuling <mikey@...ling.org>
Cc: Stewart Smith <stewart@...ux.vnet.ibm.com>
Cc: Stephane Eranian <eranian@...gle.com>
Signed-off-by: Hemant Kumar <hemant@...ux.vnet.ibm.com>

Hemant Kumar (6):
  powerpc/powernv: Data structure and macros definitions
  powerpc/powernv: Autoload IMA device driver module
  powerpc/powernv: Detect supported IMA units and its events
  powerpc/perf: Add event attribute and group to IMA pmus
  powerpc/perf: Generic ima pmu event functions
  powerpc/perf: IMA pmu cpumask and cpu hotplug support

 arch/powerpc/include/asm/ima-pmu.h             |  75 +++++
 arch/powerpc/include/asm/opal-api.h            |   3 +-
 arch/powerpc/include/asm/opal.h                |   2 +
 arch/powerpc/perf/Makefile                     |   3 +-
 arch/powerpc/perf/ima-pmu.c                    | 383 +++++++++++++++++++++
 arch/powerpc/platforms/powernv/Makefile        |   2 +-
 arch/powerpc/platforms/powernv/opal-ima.c      | 447 +++++++++++++++++++++++++
 arch/powerpc/platforms/powernv/opal-wrappers.S |   1 +
 arch/powerpc/platforms/powernv/opal.c          |  13 +
 9 files changed, 926 insertions(+), 3 deletions(-)
 create mode 100644 arch/powerpc/include/asm/ima-pmu.h
 create mode 100644 arch/powerpc/perf/ima-pmu.c
 create mode 100644 arch/powerpc/platforms/powernv/opal-ima.c

-- 
2.7.4

Powered by blists - more mailing lists