| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <CABPqkBTZa1qmw5U85F=rESsgtZ8a7OX8EmPt7KLx3OWX1KYvxA@mail.gmail.com>
Date: Fri, 27 Jan 2012 10:57:59 +0100
From: Stephane Eranian <eranian@...gle.com>
To: Anshuman Khandual <khandual@...ux.vnet.ibm.com>
Cc: linux-kernel@...r.kernel.org, peterz@...radead.org, mingo@...e.hu,
acme@...radead.org, robert.richter@....com, ming.m.lin@...el.com,
andi@...stfloor.org, asharma@...com, ravitillo@....gov,
vweaver1@...s.utk.edu
Subject: Re: [PATCH 01/13] perf_events: add generic taken branch sampling
support (v3)
On Fri, Jan 27, 2012 at 5:46 AM, Anshuman Khandual
<khandual@...ux.vnet.ibm.com> wrote:
> On Monday 09 January 2012 10:19 PM, Stephane Eranian wrote:
>> This patch adds the ability to sample taken branches to the
>> perf_event interface.
>>
>> The ability to capture taken branches is very useful for all
>> sorts of analysis. For instance, basic block profiling, call
>> counts, statistical call graph.
>>
>> This new capability requires hardware assist and as such may
>> not be available on all HW platforms. On Intel X86, it is
>> implemented on top of the Last Branch Record (LBR) facility.
>>
>> To enable taken branches sampling, the PERF_SAMPLE_BRANCH_STACK
>> bit must be set in attr->sample_type.
>>
>> Sampled taken branches may be filtered by type and/or priv
>> levels.
>>
>> The patch adds a new field, called branch_sample_type, to the
>> perf_event_attr structure. It contains a bitmask of filters
>> to apply to the sampled taken branches.
>>
>> Filters may be implemented in HW. If the HW filter does not exist
>> or is not good enough, some arch may also implement a SW filter.
>>
>> The following generic filters are currently defined:
>> - PERF_SAMPLE_USER
>> only branches whose targets are at the user level
>>
>> - PERF_SAMPLE_KERNEL
>> only branches whose targets are at the kernel level
>>
>> - PERF_SAMPLE_ANY
>> any type of branches (subject to priv levels filters)
>>
>> - PERF_SAMPLE_ANY_CALL
>> any call branches (may incl. syscall on some arch)
>>
>> - PERF_SAMPLE_ANY_RET
>> any return branches (may incl. syscall returns on some arch)
>>
>> - PERF_SAMPLE_IND_CALL
>> indirect call branches
>>
>> Obviously filter may be combined. The priv level bits are optional.
>> If not provided, the priv level of the associated event are used. It
>> is possible to collect branches at a priv level different from the
>> associated event.
>>
>> The number of taken branch records present in each sample may vary based
>> on HW, the type of sampled branches, the executed code. Therefore
>> each sample contains the number of taken branches it contains.
>>
>> Signed-off-by: Stephane Eranian <eranian@...gle.com>
> Reviewed by: Anshuman Khandual <khandual@...ux.vnet.ibm.com>
>> ---
>> arch/x86/kernel/cpu/perf_event_intel_lbr.c | 21 +++++---
>> include/linux/perf_event.h | 66 ++++++++++++++++++++++++++--
>> kernel/events/core.c | 58 ++++++++++++++++++++++++
>> 3 files changed, 133 insertions(+), 12 deletions(-)
>>
>> diff --git a/arch/x86/kernel/cpu/perf_event_intel_lbr.c b/arch/x86/kernel/cpu/perf_event_intel_lbr.c
>> index 3fab3de..c3f8100 100644
>> --- a/arch/x86/kernel/cpu/perf_event_intel_lbr.c
>> +++ b/arch/x86/kernel/cpu/perf_event_intel_lbr.c
>> @@ -144,9 +144,11 @@ static void intel_pmu_lbr_read_32(struct cpu_hw_events *cpuc)
>>
>> rdmsrl(x86_pmu.lbr_from + lbr_idx, msr_lastbranch.lbr);
>>
>> - cpuc->lbr_entries[i].from = msr_lastbranch.from;
>> - cpuc->lbr_entries[i].to = msr_lastbranch.to;
>> - cpuc->lbr_entries[i].flags = 0;
>> + cpuc->lbr_entries[i].from = msr_lastbranch.from;
>> + cpuc->lbr_entries[i].to = msr_lastbranch.to;
>> + cpuc->lbr_entries[i].mispred = 0;
>> + cpuc->lbr_entries[i].predicted = 0;
>> + cpuc->lbr_entries[i].reserved = 0;
>> }
>> cpuc->lbr_stack.nr = i;
>> }
>> @@ -167,19 +169,22 @@ static void intel_pmu_lbr_read_64(struct cpu_hw_events *cpuc)
>>
>> for (i = 0; i < x86_pmu.lbr_nr; i++) {
>> unsigned long lbr_idx = (tos - i) & mask;
>> - u64 from, to, flags = 0;
>> + u64 from, to, mis = 0, pred = 0;
>>
>> rdmsrl(x86_pmu.lbr_from + lbr_idx, from);
>> rdmsrl(x86_pmu.lbr_to + lbr_idx, to);
>>
>> if (lbr_format == LBR_FORMAT_EIP_FLAGS) {
>> - flags = !!(from & LBR_FROM_FLAG_MISPRED);
>> + mis = !!(from & LBR_FROM_FLAG_MISPRED);
>> + pred = !mis;
>> from = (u64)((((s64)from) << 1) >> 1);
>> }
>>
>> - cpuc->lbr_entries[i].from = from;
>> - cpuc->lbr_entries[i].to = to;
>> - cpuc->lbr_entries[i].flags = flags;
>> + cpuc->lbr_entries[i].from = from;
>> + cpuc->lbr_entries[i].to = to;
>> + cpuc->lbr_entries[i].mispred = mis;
>> + cpuc->lbr_entries[i].predicted = pred;
>> + cpuc->lbr_entries[i].reserved = 0;
>> }
>> cpuc->lbr_stack.nr = i;
>> }
>> diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
>> index 0b91db2..17751b1 100644
>> --- a/include/linux/perf_event.h
>> +++ b/include/linux/perf_event.h
>> @@ -129,11 +129,38 @@ enum perf_event_sample_format {
>> PERF_SAMPLE_PERIOD = 1U << 8,
>> PERF_SAMPLE_STREAM_ID = 1U << 9,
>> PERF_SAMPLE_RAW = 1U << 10,
>> + PERF_SAMPLE_BRANCH_STACK = 1U << 11,
>>
>> - PERF_SAMPLE_MAX = 1U << 11, /* non-ABI */
>> + PERF_SAMPLE_MAX = 1U << 12, /* non-ABI */
>> };
>>
>> /*
>> + * values to program into branch_sample_type when PERF_SAMPLE_BRANCH is set
>> + *
>> + * If the user does not pass priv level information via branch_sample_type,
>> + * the kernel uses the event's priv level. Branch and event priv levels do
>> + * not have to match. Branch priv level is checked for permissions.
>> + *
>> + * The branch types can be combined, however BRANCH_ANY covers all types
>> + * of branches and therefore it supersedes all the other types.
>> + */
>> +enum perf_branch_sample_type {
>> + PERF_SAMPLE_BRANCH_USER = 1U << 0, /* user level branches */
>> + PERF_SAMPLE_BRANCH_KERNEL = 1U << 1, /* kernel level branches */
>> +
>> + PERF_SAMPLE_BRANCH_ANY = 1U << 2, /* any branch types */
>> + PERF_SAMPLE_BRANCH_ANY_CALL = 1U << 3, /* any call branch */
>> + PERF_SAMPLE_BRANCH_ANY_RETURN = 1U << 4, /* any return branch */
>> + PERF_SAMPLE_BRANCH_IND_CALL = 1U << 5, /* indirect calls */
>> +
>> + PERF_SAMPLE_BRANCH_MAX = 1U << 6,/* non-ABI */
>> +};
>> +
>> +#define PERF_SAMPLE_BRANCH_PLM_ALL \
>> + (PERF_SAMPLE_BRANCH_USER|\
>> + PERF_SAMPLE_BRANCH_KERNEL)
>> +
>> +/*
>> * The format of the data returned by read() on a perf event fd,
>> * as specified by attr.read_format:
>> *
>> @@ -240,6 +267,7 @@ struct perf_event_attr {
>> __u64 bp_len;
>> __u64 config2; /* extension of config1 */
>> };
>> + __u64 branch_sample_type; /* enum branch_sample_type */
>> };
>>
>> /*
>> @@ -458,6 +486,8 @@ enum perf_event_type {
>> *
>> * { u32 size;
>> * char data[size];}&& PERF_SAMPLE_RAW
>> + *
>> + * { u64 from, to, flags } lbr[nr];} && PERF_SAMPLE_BRANCH_STACK
>> * };
>> */
>> PERF_RECORD_SAMPLE = 9,
>> @@ -530,12 +560,31 @@ struct perf_raw_record {
>> void *data;
>> };
>>
>> +/*
>> + * single taken branch record layout:
>> + *
>> + * from: source instruction (may not always be a branch insn)
>> + * to: branch target
>> + * mispred: branch target was mispredicted
>> + * predicted: branch target was predicted
>> + *
>> + * support for mispred, predicted is optional. In case it
>> + * is not supported mispred = predicted = 0.
>> + */
> So the user level perf tools would check for ((mispred = 0) && (predicted = 0))
> in a sample and report that its not supported by the HW PMU ? Point here is
> that if its not supported we should say "No HW support" rather than displaying
> mispred = 0 and predicted = 0 (As this could be misleading)
>> struct perf_branch_entry {
>> - __u64 from;
>> - __u64 to;
>> - __u64 flags;
>> + __u64 from;
>> + __u64 to;
>> + __u64 mispred:1, /* target mispredicted */
>> + predicted:1,/* target predicted */
>> + reserved:62;
>> };
>>
>> +/*
>> + * branch stack layout:
>> + * nr: number of taken branches stored in entries[]
>> + *
>> + * Note that nr can vary from sample to sample
>> + */
>> struct perf_branch_stack {
>> __u64 nr;
>> struct perf_branch_entry entries[0];
>> @@ -566,7 +615,9 @@ struct hw_perf_event {
>> unsigned long event_base;
>> int idx;
>> int last_cpu;
>> +
>> struct hw_perf_event_extra extra_reg;
>> + struct hw_perf_event_extra branch_reg;
>> };
>> struct { /* software */
>> struct hrtimer hrtimer;
>> @@ -1003,12 +1054,14 @@ struct perf_sample_data {
>> u64 period;
>> struct perf_callchain_entry *callchain;
>> struct perf_raw_record *raw;
>> + struct perf_branch_stack *br_stack;
>> };
>>
>> static inline void perf_sample_data_init(struct perf_sample_data *data, u64 addr)
>> {
>> data->addr = addr;
>> data->raw = NULL;
>> + data->br_stack = NULL;
>> }
>>
>> extern void perf_output_sample(struct perf_output_handle *handle,
>> @@ -1147,6 +1200,11 @@ extern void perf_bp_event(struct perf_event *event, void *data);
>> # define perf_instruction_pointer(regs) instruction_pointer(regs)
>> #endif
>>
>> +static inline bool has_branch_stack(struct perf_event *event)
>> +{
>> + return event->attr.sample_type & PERF_SAMPLE_BRANCH_STACK;
>> +}
>> +
>> extern int perf_output_begin(struct perf_output_handle *handle,
>> struct perf_event *event, unsigned int size);
>> extern void perf_output_end(struct perf_output_handle *handle);
>> diff --git a/kernel/events/core.c b/kernel/events/core.c
>> index 91fb68a..ed39225 100644
>> --- a/kernel/events/core.c
>> +++ b/kernel/events/core.c
>> @@ -3877,6 +3877,24 @@ void perf_output_sample(struct perf_output_handle *handle,
>> }
>> }
>> }
>> +
>> + if (sample_type & PERF_SAMPLE_BRANCH_STACK) {
>> + if (data->br_stack) {
>> + size_t size;
>> +
>> + size = data->br_stack->nr
>> + * sizeof(struct perf_branch_entry);
>> +
>> + perf_output_put(handle, data->br_stack->nr);
>> + perf_output_copy(handle, data->br_stack->entries, size);
>> + } else {
>> + /*
>> + * we always store at least the value of nr
>> + */
>> + u64 nr = 0;
>> + perf_output_put(handle, nr);
>> + }
>> + }
>> }
>>
>> void perf_prepare_sample(struct perf_event_header *header,
>> @@ -3919,6 +3937,15 @@ void perf_prepare_sample(struct perf_event_header *header,
>> WARN_ON_ONCE(size & (sizeof(u64)-1));
>> header->size += size;
>> }
>> +
>> + if (sample_type & PERF_SAMPLE_BRANCH_STACK) {
>> + int size = sizeof(u64); /* nr */
>> + if (data->br_stack) {
>> + size += data->br_stack->nr
>> + * sizeof(struct perf_branch_entry);
>> + }
>> + header->size += size;
>> + }
>> }
>>
>> static void perf_event_output(struct perf_event *event,
>> @@ -5898,6 +5925,37 @@ static int perf_copy_attr(struct perf_event_attr __user *uattr,
>> if (attr->read_format & ~(PERF_FORMAT_MAX-1))
>> return -EINVAL;
>>
>> + if (attr->sample_type & PERF_SAMPLE_BRANCH_STACK) {
>> + u64 mask = attr->branch_sample_type;
>> +
>> + /* only using defined bits */
>> + if (mask & ~(PERF_SAMPLE_BRANCH_MAX-1))
>> + return -EINVAL;
>> +
>> + /* at least one branch bit must be set */
>> + if (!(mask & ~PERF_SAMPLE_BRANCH_PLM_ALL))
>> + return -EINVAL;
>> +
>> + /* kernel level capture */
>> + if ((mask & PERF_SAMPLE_BRANCH_KERNEL)
>> + && perf_paranoid_kernel() && !capable(CAP_SYS_ADMIN))
>> + return -EACCES;
>> +
>> + /* propagate priv level, when not set for branch */
>> + if (!(mask & PERF_SAMPLE_BRANCH_PLM_ALL)) {
>> +
>> + /* exclude_kernel checked on syscall entry */
>> + if (!attr->exclude_kernel)
>> + mask |= PERF_SAMPLE_BRANCH_KERNEL;
>> +
>> + if (!attr->exclude_user)
>> + mask |= PERF_SAMPLE_BRANCH_USER;
> Why we are not taking care for attr->exclude_hv ? Should not we define
> PERF_SAMPLE_BRANCH_HV for hyper-visor level branches ?
Yes, we can add this, though I don't have any system to test it.
I will post a patch to add this priv level.
>> + /*
>> + * adjust user setting (for HW filter setup)
>> + */
>> + attr->branch_sample_type = mask;
>> + }
>> + }
>> out:
>> return ret;
>>
>
>
> --
> Linux Technology Centre
> IBM Systems and Technology Group
> Bangalore India
>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists